Integration and synchronization of input modes during multimodal human-computer interaction

DMCA

Integration and synchronization of input modes during multimodal human-computer interaction (1997)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Sharon Oviatt , Antonella Deangeli , Karen Kuhn

Citations:

177 - 20 self

BibTeX

@INPROCEEDINGS{Oviatt97integrationand,
    author = {Sharon Oviatt and Antonella Deangeli and Karen Kuhn},
    title = {Integration and synchronization of input modes during multimodal human-computer interaction},
    booktitle = {},
    year = {1997},
    pages = {415--422},
    publisher = {ACM Press}
}

OpenURL

Abstract

Our ability to develop robust multimodal systems will depend on knowledge of the natural integration patterns that typify people’s combined use of different input modes. To provide a foundadon for theory and design, the present research analyzed multimodal interaction while people spoke and wrote to a simulated dynamic map system. Task analysis revealed that multimodal interaction occurred most frequently during spatial location commands, and with intermediate tiequency during selection commands. In addition, microanalysis of input signals identitkd sequential, simultaneous, point-and-speak, and compound integration patterns, as well as data on the temporal precedence of modes and on inter-modal lags. In synchronizing input streams, the temporal pecedence of writing over speech was a major theme, with pen input conveying location information first in a sentence. Linguistic analysis also revealed that the spoken and written modes consistently supplied complementary semantic information, rather than redundant. One long-term goal of this research is the development of predictive models of natural modality integration to guide the design of emerging multimodal architectures. Keywords multimodal interaction, integration and synchronization, speech and pen input, dynamic interactive maps, spatial location information, predictive modeling

Keyphrases

multimodal human-computer interaction input mode pen input multimodal interaction multimodal architecture complementary semantic information major theme intermediate tiequency long-term goal keywords multimodal interaction temporal pecedence compound integration pattern task analysis predictive modeling spatial location command predictive model temporal precedence dynamic interactive map input stream different input mode present research input signal linguistic analysis inter-modal lag natural integration pattern spatial location information simulated dynamic map system natural modality integration location information selection command robust multimodal system

DMCA