We describe a project to capitalize on newly available levels of computational resources in order to understand human cognition. We are building an integrated physical system including vision, sound input and output, and dextrous manipulation, all controlled by a continuously operating large scale parallel MIMD computer. The resulting system will learn to "think " by building on its bodily experiences to accomplish progressively more abstract tasks. Past experience suggests that in attempting to build such an integrated system we will have to fundamentally change the way artificial intelligence, cognitive science, linguistics, and philosophy think about the organization of intelligence. We expect to be able to better reconcile the theories that will be developed with current work in neuroscience.

The emerging viewpoint of embodied cognition holds that cognitive processes are deeply rooted in the body’s interactions with the world. This position actually houses a number of distinct claims, some of which are more controversial than others. This paper distinguishes and evaluates the following six claims: 1) cognition is situated; 2) cognition is time-pressured; 3) we off-load cognitive work onto the environment; 4) the environment is part of the cognitive system; 5) cognition is for action; 6) off-line cognition is body-based. Of these, the first three and the fifth appear to be at least partially true, and their usefulness is best evaluated in terms of the range of their applicability. The fourth claim, I argue, is deeply problematic. The sixth claim has received the least attention in the literature on embodied cognition, but it may in fact be the best documented and most powerful of the six claims.

Concepts are the elementary units of reason and linguistic meaning. They are conventional and relatively stable. As such, they must somehow be the result of neural activity in the brain. The questions are: Where? and How? A common philosophical position is that all concepts—even concepts about action and perception—are symbolic and abstract, and therefore must be implemented outside the brain’s sensory-motor system. We will argue against this position using (1) neuroscientific evidence; (2) results from neural computation; and (3) results about the nature of concepts from cognitive linguistics. We will propose that the sensory-motor system has the right kind of structure to characterise both sensory-motor and more abstract concepts. Central to this picture are the neural theory of language and the theory of cogs, according to which, brain structures in the sensory-motor regions are exploited to characterise the so-called “abstract ” concepts that constitute the meanings of grammatical constructions and general inference patterns.

Visual thinking plays an important role in scientific reasoning. Based on the research in automating diverse reasoning tasks about dynamical systems, nonlinear controllers, kinematic mechanisms, and fluid motion, we have identified a style of visual thinking, imagistic reasoning. Imagistic reasoning organizes computations around image-like, analogue representations so that perceptual and symbolic operations can be brought to bear to infer structure and behavior. Programs incorporating imagistic reasoning have been shown to perform at an expert level in domains that defy current analytic or numerical methods. We have developed a computational paradigm, spatial aggregation, to unify the description of a class of imagistic problem solvers. A program written in this paradigm has the following properties. It takes a continuous field and optional objective functions as input, and produces high-level descriptions of structure, behavior, or control actions. It computes a multi-layer of interme...

Three theories currently compete to explain the conceptual deficits that result from brain damage: sensory-functional theory, domain-specific theory, and conceptual structure theory. We argue that all three theories capture important aspects of conceptual deficits, and offer different insights into their origins. Conceptual topography theory (CTT) integrates these insights, beginning with A. R. Damasio’s (1989) convergence zone theory and elaborating it with the similarity-in-topography (SIT) principle. According to CTT, feature maps in sensory-motor systems represent the features of a category’s exemplars. A hierarchical system of convergence zones then conjoins these features to form both property and category representations. According to the SIT principle, the proximity of two conjunctive neurons in a convergence zone increases with the similarity of the features they conjoin. As a result, conjunctive neurons become topographically organised into local regions that represent properties and categories. Depending on the level and location of a lesion in this system, a wide variety of deficits is possible. Consistent with the literature, these deficits range from the loss of a single category to the loss of multiple categories that share sensory-motor properties.

While the mind remains a mysterious and inaccessible phenomenon, many of the components of mind, such as perception, behavior generation, knowledge representation, value judgment, reason, intention, emotion, memory, imagination, recognition, learning, attention, and intelligence are becoming well defined and amenable to analysis. Progress is rapid in the cognitive and neurosciences as well as in artificial intelligence, control theory, and many other fields related to the engineering of mind. A reference model architecture for intelligent systems is suggested to tie together concepts from all these separate fields into a unified framework that includes both biological and machine embodiments of the components of mind. It is argued that such a reference model architecture will facilitate the development of scientific models of mind. 1. Introduction What is mind? What is the relationship between the mind and the brain? What is thought? What are the mechanisms that give ...

Self-explaining is an effective metacognitive strategy that can help learners develop deeper understanding of the material they study. This experiment explored if the format of material (i.e., text or diagrams) influences the self-explanation effect. Twenty subjects were presented with information about the human circulatory system and prompted to self-explain; 10 received this information in text and 10 in diagrams. Results showed that students given diagrams performed significantly better on post-tests than students given text. Diagrams students also generated significantly more self-explanations that text students. Furthermore, the benefits of self-explaining were much greater in the diagrams condition. To discover why diagrams can promote the self-explanation effect, results are interpreted with reference to the multiple differences in the semantic, cognitive and affective properties of the texts and diagrams studied.

There is widespread anecdotal evidence that expert programmers make use of visual mental images when they are designing programs. This evidence is used to justify the use of diagrams and visual programming languages during software design. This paper reports the results of two studies. In the first, expert programmers were directly questioned regarding the nature of their mental representations while they were engaged in a design task. This investigative technique was used with the explicit intention of eliciting introspective reports of mental imagery. In the second, users of a visual programming language responded to a questionnaire in which they were asked about cognitive processes. The resulting transcripts displayed a considerable number of common elements. These suggest that software design shares many characteristics of more concrete design disciplines. The reports from participants in the two studies, together with previous research into imagery use, indicate potential...

Below is the unedited, uncorrected final draft of a BBS target article that has been accepted for publication. This preprint has been prepared for potential commentators who wish to nominate themselves for formal commentary invitation. Please DO NOT write a commentary until you receive a formal invitation. If you are invited to submit a commentary, a copyedited, corrected version of this paper will be posted.

Computational models of analogical problem solving have traditionally described source and target domains in terms of their causal structure. But psychological research shows that visual reasoning plays a part for many kinds of analogies. This paper describes a model that transfers a solution from a source analog to a new target problem using only visual knowledge represented symbolically. The knowledge representation is based on a language of primitive visual elements and transformations. We found that visual knowledge is sufficient for transfer, but that causal knowledge is needed to determine if the transferred solution is appropriate. 1