Abstract not found

1 Language in the World How does language relate to the non-linguistic world? If an agent is able to communicate linguistically and is also able to directly perceive and/or act on the world, how do perception, action, and language interact with and influence each other? Such questions are surely amongst the most important in Cognitive Science and Artificial Intelligence (AI). Language, after all, is a central aspect of the human mind – indeed it may be what distinguishes us from other species. There is sometimes a tendency in the academic world to study language in isolation, as a formal system with rules for well-constructed sentences; or to focus on how language relates to formal notations such as symbolic logic. But language did not evolve as an isolated system or as a way of communicating symbolic logic; it presumably evolved as a mechanism for exchanging information about the world, ultimately providing the medium for cultural transmission across generations. Motivated by these observations, the goal of this special issue is to bring together research in AI that focuses on relating language to the physical world. Language is of course also used to communicate about non-physical referents, but the ubiquity of physical metaphor in language [21] suggests that grounding in the physical world provides the foundations of semantics.

The Neural Theory of Language project aims to build structured connectionist models of language and cognition consistent with constraints from all domains and at all levels. These constraints include recent experimental evidence that details of neural computation and brain architecture play a crucial role in language processing. We focus in this paper on the computational level and explore the role of embodied representations and simulative inference in language understanding. 1.

We present a structured model of context that supports an integrated approach to language acquisition and use. The model extends an existing formal notation, Embodied Construction Grammar (ECG), with representations for tracking both entities and events in discourse and situational context. The notation employs an intermediate level of granularity between low-level sensorimotor representations (such as that suitable for dynamic models of action and events for grounded language learning) and the more schematic representations needed for learning and using grammar. The resulting model allows existing systems for simulation-based language understanding and comprehension-driven grammar learning to represent, interpret and acquire a variety of contextually grounded construction.

Running Head: The origin of epistemic structures Abstract: Organisms across species use the strategy of generating structures in their environment to lower cognitive complexity. Examples include pheromones, markers, colour codes, etc. Distributed Cognition theory has argued that studying such ‘epistemic structures ’ can provide insights into the development and nature of internal representations, and cognition itself. We develop this claim by providing a model of the origin of such structures, and present a simulation where organisms with only reactive behaviour learn, within their lifetime, to add such structures systematically to their world to lower cognitive load. This implementation is then extended to show that the same underlying process could generate traces of the world in an ‘internal environment ’ to lower cognitive load. We then examine two implications of this internal trace model. First, it provides a novel account of the origin of internal representations. Further, as both external and internal traces lower cognitive load and are generated using the same mechanism, the location of the structure becomes opportunistic, and a matter of utility. This supports the ‘extended mind ’ hypothesis. Second, the stored internal traces develop entirely out of actions. They thus encapsulate action components and could activate actions. This feature explains the origin of enactable and action-oriented mental content.

Many models of the acquisition of early multi-word constructions assume significant

There is mounting evidence that language comprehension involves the activation of mental imagery

We take up the challenge of learning a grounded model of language when our agent has a body of machine learning algorithms and no prior knowledge of either the physical domain or language, in the sense of "least commitment". Based on a 2D video and co-occurring raw text, we demonstrate how this cognitively inspired model segments the world to obtain a meaning space, and combines words into hierarchical patterns for a linguistic pattern space. By associating these two spaces under temporal co-occurrence constraints, we demonstrate the acquisition of term-meaning pairs for names, actions and relations. We next map physical arguments for actions and relations to syntactical constructions resembling a cognitive grammar framework. Thus the system is able to bootstrap a rudimentary lexicon and syntax. While experiments are primarily in English, we present partial results for Hindi obtained without any change in the methods, to indicate its potential application to other languages.