DMCA
Toward Multimodal Fusion of Affective Cues (2006)
| Citations: | 8 - 1 self |
BibTeX
@MISC{Paleari06towardmultimodal,
author = {Marco Paleari and Christine L. Lisetti},
title = { Toward Multimodal Fusion of Affective Cues},
year = {2006}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
During face to face communication, it has been suggested that as much as 70 % of what people communicate when talking directly with others is through paralanguage involving multiple modalities combined together (e.g. voice tone and volume, body language). In an attempt to render humancomputer interaction more similar to human-human communication and enhance its naturalness, research on sensory acquisition and interpretation of single modalities of human expressions have seen ongoing progress over the last decade. These progresses are rendering current research on artificial sensor fusion of multiple modalities an increasingly important research domain in order to reach better accuracy of congruent messages on the one hand, and possibly to be able to detect incongruent messages across multiple modalities (incongruency being itself a message about the nature of the information being conveyed). Accurate interpretation of emotional signals- quintessentially multimodal- would hence particularly benefit from multimodal sensor fusion and interpretation algorithms. In this paper we provide a state of the art multimodal fusion and describe one way to implement a generic framework for multimodal emotion recognition. The system is developed within the MAUI framework [31] and Scherer’s Component Process Theory (CPT) [49, 50, 51, 24, 52], with the goal to be modular and adaptive. We want the designed framework to be able to accept different single and multi modality recognition systems and to automatically adapt the fusion algorithm to find optimal solutions. The system also aims to be adaptive to channel (and system) reliability.
Keyphrases
affective cue multiple modality abstract toward multimodal fusion current research generic framework accurate interpretation human expression maui framework single modality body language ongoing progress congruent message art multimodal fusion multimodal emotion recognition emotional signal human-human communication incongruent message multimodal sensor fusion multi modality recognition system optimal solution component process theory last decade interpretation algorithm important research domain sensory acquisition artificial sensor fusion voice tone fusion algorithm
