ABSTRACT: This paper presents a model of how hippocampal place cells might be used for spatial navigation in two watermaze tasks: the standard reference memory task and a delayed matching-to-place task. In the reference memory task, the escape platform occupies a single location and rats gradually learn relatively direct paths to the goal over the course of days, in each of which they perform a fixed number of trials. In the delayed matching-to-place task, the escape platform occupies a novel location on each day, and rats gradually acquire one-trial learning, i.e., direct paths on the second trial of each day. The model uses a local, incremental, and statistically efficient connectionist algorithm called temporal difference learning in two distinct components. The first is a reinforcement-based ‘‘actor-critic’ ’ network that is a general model of classical and instrumental conditioning. In this case, it is applied to navigation, using place cells to provide information about state. By itself, the actor-critic can learn the reference memory task, but this learning is inflexible to changes to the platform location. We argue that one-trial learning in the delayed matching-to-place task demands a goal-independent representation of space. This is provided by the second component of the model: a network that uses temporal difference learning and selfmotion information to acquire consistent spatial coordinates in the environment. Each component of the model is necessary at a different stage of the task; the actor-critic provides a way of transferring control to the component that performs best. The model successfully captures gradual acquisition in both tasks, and, in particular, the ultimate development of one-trial learning in the delayed matching-to-place task. Place cells report a form of stable, allocentric information that is well-suited to the various kinds of learning in the model. Hippocampus 2000;10:1–16.

Our model of the spatial and temporal aspects of place cell firing, and their role in rat navigation is reviewed. The model provides a can- didate mechanism, at the level of individual cells, by which place cell information concerning self-localization could be used to guide navi- gation to previously visited reward sites. The model embodies specific predictions regarding the formation of place fields, the phase coding of place cell firing with respect to the hippocampal theta rhythm, and the formation of neuronal population vectors downstream from the place cells that code for the directions of goals during navigation. Re- cent experiments regarding the spatial distribution of place cell firing have confirmed our initial modeling hypothesis, that place fields are formed from Gaussian tuning curve inputs coding for the distances from environmental features, and enabled us to further specify the functional form of these inputs. Other recent experiments regarding the temporal distribution of place cell firing in 2-dimensional environ- ments have confirmed our predictions based on the temporal aspects of place cell firing on linear tracks. Directions for further experiments and refinements to the model are outlined for the future.