Abstract. The Text2Dialogue (T2D) system that we are developing allows digital content creators to generate attractive multi-modal dialogues presented by two virtual agents—by simply providing textual information as input. We use Rhetorical Structure Theory (RST) to decompose text into segments and to identify rhetorical discourse relations between them. These are then “acted out ” by two 3D agents using synthetic speech and appropriate conversational gestures. In this paper, we present version 1.0 of the T2D system and focus on the novel technique that it uses for mapping rhetorical relations to question–answer pairs, thus transforming (monological) text into a form that supports dialogues between virtual agents. 1

amounts of patient data in various formats, making efficient processing of information by medical professionals difficult. Moreover, different stakeholders in the neonatal scenario, which include parents as well as staff occupying different roles, have different information requirements. This paper describes recent and ongoing work on building systems that automatically generate textual summaries of neonatal data. Our evaluation results show that the technology is viable and comparable in its effectiveness for decision support to existing presentation modalities. We discuss the lessons learned so far, as well as the major challenges involved in extending current technology to deal with a broader range of data types, and to improve the textual output in the form of more coherent summaries.

Abstract. In this paper we describe the graphical realization and authoring of a semi-autonomous animated persona, called a Digital Puppet. Digital Puppets augment on-line educational materials with assistance, commentary and summaries, and have been used in both the tutoring and presentation domains. Limited animation, 2D "cartoon " characters are useful for some pedagogical applications, particularly for young learners, but production demands frequently exceed limited research budgets. To make the production of 2D personas feasible, we present a build-once, use-forever production path that allows us to quickly create new behaviors for characters. Animations are generated by selecting behaviors based on textual content and presentation and tutorial goals, allowing the puppet to act with limited autonomy within specific domains. 1

Abstract. In this study, we argue that multimodal presentations should be created in a cognitive-aware manner, especially in a high-load HCI situation where the user task challenges the full capacity of the human cognition. An experiment was conducted to investigate the cognitive effects of modalities, using a high-load task. The performance measurements and subjective reports consistently confirm a significant modality impact on cognitive workload, stress and performance. A relation between modality usage and physiological states was not found, due to the insufficient sensitivity and individual differences of the physiological measurements. The findings of this experiment can be well explained by several modality-related cognitive theories. We further integrate these theories into a suitability prediction model, which can systematically predict how suitable a certain modality usage is for this presentation task. The model demonstrates a possible approach towards cognitive-aware modality planning and can be modified for other applications. Keywords: Cognitive-aware, multimodal presentation, modality planning, cognitive load, stress, performance, high-load HCI. 1

In this study, a user experiment was conducted to investigate the effects of information presentation factors (modality and structure) on decision making behavior, using a time-limited task. The time constraint required subjects to develop heuristic strategies to substitute the defined normative strategy. The two presentation factors have been shown to significantly affect the decision making performance, assessed by time efficiency and accuracy. The modality factor mainly influenced the time efficiency, due to its impact on the efficiency of information perception. By analyzing the subjective reports and the error distribution, the structure was shown to influence the selection of heuristic strategies. Consequentially, it affected both the time efficiency and the accuracy of decision making. The interaction between the time constraint and the presentation effects was also observed.

making with a time limit:

Abstract Intelligent multimodal presentation (IMMP) systems are able to generate multimodal presentations adaptively, based on the run-time requirements of user–computer interaction. Modality allocation in IMMP system needs to adapt the modality choice to changes in various relevant factors, such as the type of information to be conveyed, the presentation goal, the characteristics of the available modalities, the user profile, the condition of the environment, and the type of user task. In this study, we emphasize that modality allocation in IMMP systems should also take into account the cognitive impacts of modality on human information processing. We first describe several modality-related cognitive and neuropsychological findings. Then a user study is presented to demonstrate the effects of modality on performance, cognitive load and stress, using a high-load and time-critical user task. Finally, we show a possible way to integrate relevant cognitive theories into a computational model that can systematically predict the suitability of a modality choice for a given presentation task. 1

For guidance on citations see FAQs. c ○ [not recorded] Version: [not recorded] Link(s) to article on publisher’s website: