DMCA
Optimization in Multimodal Interpretation (2004)
Cached
Download Links
| Venue: | In Proceedings of 42nd Annual Meeting of Association for Computational Linguistics (ACL |
| Citations: | 8 - 4 self |
BibTeX
@INPROCEEDINGS{Chai04optimizationin,
author = {Joyce Y. Chai and Pengyu Hong and Michelle X. Zhou and Zahar Prasov},
title = {Optimization in Multimodal Interpretation},
booktitle = {In Proceedings of 42nd Annual Meeting of Association for Computational Linguistics (ACL},
year = {2004},
pages = {1--8}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
In a multimodal conversation, the way users communicate with a system depends on the available interaction channels and the situated context (e.g., conversation focus, visual feedback). These dependencies form a rich set of constraints from various perspectives such as temporal alignments between different modalities, coherence of conversation, and the domain semantics. There is strong evidence that competition and ranking of these constraints is important to achieve an optimal interpretation. Thus, we have developed an optimization approach for multimodal interpretation, particularly for interpreting multimodal references. A preliminary evaluation indicates the effectiveness of this approach, especially for complex user inputs that involve multiple referring expressions in a speech utterance and multiple gestures. 1
Keyphrases
multimodal interpretation speech utterance preliminary evaluation various perspective optimal interpretation optimization approach available interaction channel domain semantics way user complex user input multimodal reference different modality strong evidence visual feedback multimodal conversation conversation focus temporal alignment multiple gesture rich set
