Architecture of the Dialogue Shell ***DRAFT*** 2/00 to appear in Natural Language Engineering, 2000. 7 mantic hierarchy and to a world KB manager that handles queries about the current situation, managing the interfaces to domain dependent reasoners and knowledge bases as needed. One of the key things to note about this architecture is the separation of the basic dialogue system components from the more domain-specific components that provide the application (shown within the dotted lines at the lower left corner of Figure 1). To illustrate this separation, consider a specific example: a travel-agent application. The back-end would provide schedule and reservation information, booking, and so on, much as current computer systems provide to human travel agents. The behavioral agent and plan manager would be driven from a specification of desired behavior of the system as a travel agent, including the actions it typically will be asked to perform (e.g., what information is relevant to ...

Traditionally, research in applying Semantic Web technology to multimedia information systems has focused on using annotations and ontologies to improve the retrieval process. This paper concentrates on improving the presentation of the retrieval results.

Automatic multimedia presentation generation is applicable in a wide variety of circumstances because of its ability to adapt to di#erent presentation contexts such as hardware platforms, user expertise and user interest. The process

Abstract. The automated generation of multimedia reports for timevarying scenes on the basis of visual data constitutes a challenging research goalwith a high potential for many interesting applications. In this paper, we report on our work towards an automatic commentator system for RoboCup, the Robot World-Cup Soccer. Rocco (RoboCup-Commentator) is a prototype system that has emerged from our previous work on high-level scene analysis and intelligent multimedia generation. Based on a general conception for multimedia reporting systems, we describe the initial Rocco version which isintended to generate TV-style live reports for matches of the simulator league. 1

Abstract. Applications that use annotated multimedia assets need to be able to process all the annotations about a specific media asset. At first sight, this seems almost trivial, but annotations are needed for different levels of description, these need to be related to each other in the appropriate way and, in particular on the Semantic Web, annotations may not all be stored in the same place. We distinguish between technical descriptions of a media asset from content-level descriptions. At both levels, the annotations needed in a single application may come from different vocabularies. In addition, the instantiated values for a term used from an ontology also need to be specified. We present a number of existing vocabularies related to multimedia, discuss the above problems then discuss requirements for and the desirability of a lightweight multimedia ontology. 1

Multimedia systems—systems which employ several media such as text, graphics, animation and sound for the presentation of information—have become widely available during the last decade. The acceptance and usability of such systems is, however, substantially affected by their limited ability to present information in a flexible manner. As the need for flexibity grows, the manual creation of multimedia presentations becomes less and less feasible. While the automatic production of material for presentation is rarely addressed in the multimedia community, a considerable amount of research effort has been directed towards the automatic generation of natural language. The purpose of this chapter is to introduce techniques for the automatic production of multimedia presentations; these techniques draw upon lessons learned during the development of natural language generators.

Abstract. This paper presents an approach to assist authors during the authoring of multimedia presentations. We extend existing authoring support by integrating processes of topic identification, content collection and discourse structure building in a single environment. This integration allows identification of the context of the authoring process. Our approach combines this process context awareness with explicit domain and discourse knowledge to steer system suggestions. We evaluate our approach with an experimental system prototype.

The development of large-scale dialog systems requires a flexible architecture model and adequate software support to cope with the challenge of system integration. This contribution presents a general framework for building integrated natural-language and multimodal dialog systems. Our approach relies on a distributed component model that enables flexible re-use and extension of existing software modules and is able to deal with a heterogeneous software environment. A practical result of our research is the development of a sophisticated integration platform, called MULTIPLATFORM, which is based on the proposed framework. This MULTIPLATFORM testbed has been used in various large and mid-size research projects to develop integrated system prototypes.

We present the rags (Reference Architecture for Generation Systems) framework: a specification of an abstract Natural Language Generation (NLG) system architecture to support sharing, re-use, comparison and evaluation of NLG technologies. We argue that the evidence from a survey of actual NLG systems calls for a different emphasis in a reference proposal from that seen in similar initiatives in information extraction and multimedia interfaces. We introduce the framework itself, in particular the two-level data model that allows us to support the complex data requirements of NLG systems in a flexible and coherent fashion, and describe our efforts to validate the framework through a range of implementations.

We present a new approach for adaptive presentation of structured information, based on preference-based constrained optimization techniques rooted in qualitative decisiontheory.