Learning Visually-Grounded Words and Syntax for a Scene Description Task (0)
by
Deb K. Roy
| Citations: | 30 - 16 self |
BibTeX
@MISC{Roy_learningvisually-grounded,
author = {Deb K. Roy},
title = {Learning Visually-Grounded Words and Syntax for a Scene Description Task},
year = {}
}
Years of Citing Articles
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
A spoken language generation system has been developed that learns to describe objects in computer-generated visual scenes. The system is trained by a `show-and-tell' procedure in which visual scenes are paired with natural language descriptions. Learning algorithms acquire probabilistic structures which encode the visual semantics of phrase structure, word classes, and individual words. Using these structures, a planning algorithm integrates syntactic, semantic, and contextual constraints to generate natural and unambiguous descriptions of objects in novel scenes.
