Abstract. We present a dialogue architecture that addresses perception, planning and execution of multimodal dialogue behavior. Motivated by realtime human performance and modular architectural principles, the architecture is full-duplex (“open-mic”); prosody is continuously analyzed and used for mixed-control turntaking behaviors (reactive and deliberative) and incremental utterance production. The architecture is fine-grain and highly expandable; we are currently applying it in more complex multimodal interaction and dynamic task environments. We describe here the theoretical underpinnings behind the architecture, compare it to prior efforts, discuss the methodology and give a brief overview of its current runtime characteristics.

The general intelligence of any autonomous system must in large part be measured by its ability to automatically learn new skills and integrate these with prior skills. Cognitive architectures addressing these topics are few and far between – possibly because of their difficulty. We argue that architectures capable of diverse skill acquisition and integration, and real-time management of these, require an approach of modularization that goes well beyond the current practices, leading to a class of architectures we refer to as peewee-granule systems. The building blocks (modules) in such systems have simple operational semantics and result in architectures that are heterogeneous at the cognitive level but homogeneous at the computational level.

Aims and scope of the series During the past decade computer science research in understanding and reproducing human intelligence has expanded from the more traditional approaches like psychology, logics and artificial intelligence into multiple other areas, including neuroscience research. Moreover, new results in biology, chemistry, (surface) physics and gene technology, but also in network technology are greatly affecting current research in computer science, including the development of intelligent systems. At the same time, computer science’s new results are increasingly being applied in these fields allowing for important crossfertilisations. This series aims at publishing proceedings from all disciplines dealing with and affecting the issue of understanding and reproducing intelligence in artificial systems. Also, the series is open for publications concerning the application of intelligence in networked or any other environment and the extraction of meaningful data from large data sets.