We develop a theoretical framework that shows how mesencephalic dopamine systems could distribute to their targets a signal that represents information about future expectations. In particular, we show how activity in the cerebral cortex can make predictions about future receipt of reward and how fluctuations in the activity levels of neurons in diffuse dopamine systems above and below baseline levels would represent errors in these predictions that are delivered to cortical and subcottical targets. We present a model for how such errors could be constructed in a real brain that is consistent with physiological results for a subset of dopaminergic neurons located in the ventral tegmental area and surrounding dopaminergic neurons. The theory also makes testable predictions about human choice behavior on a simple decision-making task. Furthermore, we show that, through a simple influence on synaptic plasticity, fluctuations in dopamine release can act to change the predictions in an appropriate manner. Key words: prediction; dopamine; diffuse ascending systems; synaptic plasticity; reinforcement learning; reward In mammals, mesencephalic dopamine neurons participate in a number of important cognitive and physiological functions including motivational processes (Wise, 1982; Fibiger and Phillips, 1986; Koob and Bloom, 1988) reward processing (Wise, 1982) working

The head-direction (HD) cells found in the limbic system in freely moving rats represent the instantaneous head direction of the animal in the horizontal plane regardless of the location of the animal. The internal direction represented by these cells uses both self-motion information for inet-tially based updating and familiar visual landmarks for calibration. Here, a model of the dynamics of the HD cell ensemble is presented. The sta-bility of a localized static activity profile in the network and a dynamic shift mechanism are explained naturally by synaptic weight distribution components with even and odd symmetry, respectively. Under symmetric weights or symmetric reciprocal connections, a stable activity profile close to the known direc-tional tuning curves will emerge. By adding a slight asymmetry to the weights, the activity profile will shift continuously without 1

ABSTRACT: This paper presents a model of how hippocampal place cells might be used for spatial navigation in two watermaze tasks: the standard reference memory task and a delayed matching-to-place task. In the reference memory task, the escape platform occupies a single location and rats gradually learn relatively direct paths to the goal over the course of days, in each of which they perform a fixed number of trials. In the delayed matching-to-place task, the escape platform occupies a novel location on each day, and rats gradually acquire one-trial learning, i.e., direct paths on the second trial of each day. The model uses a local, incremental, and statistically efficient connectionist algorithm called temporal difference learning in two distinct components. The first is a reinforcement-based ‘‘actor-critic’ ’ network that is a general model of classical and instrumental conditioning. In this case, it is applied to navigation, using place cells to provide information about state. By itself, the actor-critic can learn the reference memory task, but this learning is inflexible to changes to the platform location. We argue that one-trial learning in the delayed matching-to-place task demands a goal-independent representation of space. This is provided by the second component of the model: a network that uses temporal difference learning and selfmotion information to acquire consistent spatial coordinates in the environment. Each component of the model is necessary at a different stage of the task; the actor-critic provides a way of transferring control to the component that performs best. The model successfully captures gradual acquisition in both tasks, and, in particular, the ultimate development of one-trial learning in the delayed matching-to-place task. Place cells report a form of stable, allocentric information that is well-suited to the various kinds of learning in the model. Hippocampus 2000;10:1–16.

This paper argues for the development of synthetic approaches towards the study of brain and behavior as a complement to the more traditional empirical mode of research. As an example we present our own work on learning and problem solving which relates to the behavioral paradigms of classical and operant conditioning. We de ne the concept of learning in the context of behavior and lay out the basic methodological requirements a model needs to satisfy, which includes evaluations using robots. In addition, we de ne a number of design principles neuronal models should obey to be considered relevant. We present in detail the construction of a neural model of short- and long-term memory which can be applied to an arti cial behaving system. The presented model (DAC4) provides a novel self-consistent implementation of these processes, which satis es our principles. This model will be interpreted towards the present understanding of the neuronal substrate of memory.

ABSTRACT: We describe some of the results of our program of basic and applied research on navigating without vision. One basic research topic that we have studied extensively is path integration, a form of navigation in which perceived self-motion is integrated over time to obtain an estimate of current posilion and orientation. In experiments on pathway completion, one test of path integration ability, we have found that subjects who are passively guided over the outbound path without vision exhibit significant errors when attempting to return to the origin but are nevertheless sensitive to turns and segment lengths in the stimulus path. We have also found no major differences in path inlegration ability among blirid and sighted populations. A model we havc developed that attributes errors in path integration to errors in encoding the stimulus path is a good beginning toward understanding path integration performance. In otber research on path integration, in which optic flow information was manipulated in addition to the proprioceptive and vestibular information of nonvisual locomotion, we havc found that optic flow is a weak input to the path integration process. In other basic research, our studies of auditory distance perception in outdoor environments show systematic underestimation oC sound source distance. Our applied research has been concerned with developing and evaluating a navigation system for the visually impaired that uses three recent technologies: the Global Positioning System, Geographic Information Systems, and virtual acouslics. Our work shows that there is considerable promise of these three technologies in allowing visually impaired individuals to navigate and learn about unfamiliar environments without the assistance of human guides. (Optoni Vis Sci 2001;78:282-289)

Abstract not found

Vestibular information influences spatial orientation and navigation in laboratory animals and humans. Neurons within the rat anterior thalamus encode the directional heading of the animal in absolute space. These neurons, referred to as head direction (HD) cells, fire selectively when the rat points its head in a specific direction in the horizontal plane with respect to the external laboratory reference frame. HD cells are thought to represent an essential component of a neural network that processes allocentric spatial information. The functional properties of HD cells may be dependent on vestibular input. Here, anterior thalamic HD cells were recorded before and after sodium arsanilate-induced vestibular system lesion. Vestibular lesions abolished the directional firing properties of HD cells. The time course of disruption in the directional firing properties paralleled the loss of vestibular function. Arsanilate-treated rats

Many neurons in the rat lateral mammillary nuclei (LMN) fire selectively in relation to the animal’s head direction (HD) in the horizontal plane independent of the rat’s location or behavior. One hypothesis of how this representation is generated and updated is via subcortical projections from the dorsal tegmental nucleus (DTN). Here we report the type of activity in DTN neurons. The majority of cells (75%) fired as a function of the rat’s angular head velocity (AHV). Cells exhibited one of two types of firing patterns: (1) symmetric, in which the firing rate was positively correlated with AHV during head turns in both directions, and (2) asymmetric, in which the firing rate was positively correlated with head turns in one direction and correlated either negatively or not at all in the opposite direction. In addition to modulation by AHV, some of the AHV cells (40.1%) Neurons that discharge selectively in relation to an animal’s head

Jeffrey S. Taube. Passive transport disrupts directional path integration by rat head direction cells. J Neurophysiol 90: 2862–2874, 2003. First published July 30, 2003; 10.1152/jn.00346.2003. A subset of neurons in the rat limbic system encodes head direction (HD) by selectively discharging when the rat points its head in a preferred direction in the horizontal plane. The preferred firing direction is sensitive to the location of landmark cues, as well as idiothetic or self-motion cues (i.e., vestibular, motor efference copy, proprioception, and optic flow). Previous studies have shown that the preferred firing direction remains relatively stable (average shift � 18°) after the rat walks from a familiar environment into a novel one, suggesting that without familiar landmarks, the preferred firing direction can be maintained using idiothetic cues, a process called directional path integration. This study repeated this experiment and manipulated the idiothetic cues available to the rat as it moved between the familiar and novel environment. Motor efference copy/proprioceptive cues were disrupted by passively transporting the animal between the familiar and novel environment. Darkening the room as the animal moved to the novel environment eliminated optic flow cues. HD cell preferred firing directions shifted in the novel environment by an average of 30 ° after locomotion from the familiar environment with the room lights off; by an average of 70 ° after passive transport from the familiar environment with the room lights on; and by an average of 67 ° after passive transport with the room lights off. These findings are consistent with the view that motor efference copy/proprioception cues are important for maintaining the preferred firing direction of HD cells under conditions requiring path integration.

ABSTRACT: The spatial mapping theory of hippocampal function proposes that the rat hippocampus is specialized for navigational computations, computations that allow the animal to solve difficult spatial problems. In this paper, we review evidence obtained by recording place cells and other ‘‘spatially tuned’ ’ cells from freely moving rats. Our main conclusion is that the nature of the signals carried by these cells and the ways in which the signals transform after changing the environment imply that the hippocampus and associated structures are able to represent aspects of the geometry of the environment. Hippocampus 1999; 9:413–422. � 1999 Wiley-Liss, Inc. The ability of rats and mice to solve difficult navigational problems implies that they can form and use representations of their environment to select paths from their initial location to a goal (Tolman, 1948; Gallistel, 1990; Poucet, 1993). There are two very different ways in which such representations could be formed in the nervous system. On the one hand,