DMCA
Multimodal Saliency-based Attention for Object-based Scene Analysis
Cached
Download Links
| Citations: | 6 - 4 self |
BibTeX
@MISC{Schauerte_multimodalsaliency-based,
author = {Boris Schauerte and Benjamin Kühn and Kristian Kroschel and Rainer Stiefelhagen},
title = {Multimodal Saliency-based Attention for Object-based Scene Analysis},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract — Multimodal attention is a key requirement for humanoid robots in order to navigate in complex environments and act as social, cognitive human partners. To this end, robots have to incorporate attention mechanisms that focus the processing on the potentially most relevant stimuli while controlling the sensor orientation to improve the perception of these stimuli. In this paper, we present our implementation of audio-visual saliency-based attention that we integrated in a system for knowledge-driven audio-visual scene analysis and object-based world modeling. For this purpose, we introduce a novel isophote-based method for proto-object segmentation of saliency maps, a surprise-based auditory saliency definition, and a parametric 3-D model for multimodal saliency fusion. The applicability of the proposed system is demonstrated in a series of experiments. Index Terms — audio-visual saliency, auditory surprise, isophote-based visual proto-objects, parametric 3-D saliency model, object-based inhibition of return, multimodal attention, scene exploration, hierarchical object analysis, overt attention, active perception I.
Keyphrases
multimodal saliency-based attention object-based scene analysis complex environment sensor orientation saliency map cognitive human partner object-based world modeling multimodal saliency fusion proposed system surprise-based auditory saliency definition knowledge-driven audio-visual scene analysis hierarchical object analysis isophote-based visual proto-objects scene exploration abstract multimodal attention audio-visual saliency-based attention novel isophote-based method object-based inhibition multimodal attention index term audio-visual saliency parametric 3-d saliency model proto-object segmentation auditory surprise key requirement humanoid robot attention mechanism parametric 3-d model active perception overt attention relevant stimulus
