DMCA
Multimodality In Language And Speech Systems - From Theory To Design Support Tool (2001)
| Citations: | 43 - 15 self |
BibTeX
@MISC{Bernsen01multimodalityin,
author = {Niels Ole Bernsen},
title = {Multimodality In Language And Speech Systems - From Theory To Design Support Tool},
year = {2001}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Advances in information technologies and telecommunications are making available a very large number of possible input/output modality combinations which are potentially useful for the expression and exchange of information in human-human-system interaction (HHSI). However, standard or shared foundations consisting of all available and possible elementary, or unimodal, input and output modalities is still lacking. A common representation format for describing the relevant properties of individual unimodal modalities is lacking as well. Ideally, such foundations could enable designers of multimodal interaction to select an appropriate set of modalitie s for an application to be designed, once its information representation and exchange requirements have been specified. This chapter proposes Modality Theory-based solutions and discusses alternative approaches to major parts of this practical problem of modality selection. The solutions cover the generation, at descending levels of abstraction, of taxonomies of unimodal input and output modal ities from basic properties in the media of graphics, acoustics and haptics. The result is a practical `designer's t oolbox' of unimodal modalities. A first common representation format for unimodal modalities is presented. The taxonomy and the unimodal modalities as represented in the common format are then brought to the test by addressing the complex issue of when (not) to use speech in practical applications. Finally, empirical and theoretical approaches to the combinatorial explosion of modality combinations in multimodal systems are discussed.
Keyphrases
unimodal modality speech system theory design support tool multimodal system modality theory-based solution output modality modality selection common representation format large number major part output modal ities theoretical approach practical problem human-human-system interaction practical application multimodal interaction appropriate set relevant property complex issue exchange requirement first common representation format modality combination discus alternative approach information technology possible input output modality combination basic property common format individual unimodal modality combinatorial explosion unimodal input information representation
