DMCA
An Extended Pose-Invariant Lipreading System
| Citations: | 5 - 1 self |
BibTeX
@MISC{Lucey_anextended,
author = {Patrick Lucey and Gerasimos Potamianos and Sridha Sridharan},
title = {An Extended Pose-Invariant Lipreading System},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
In recent work, we have concentrated on the problem of lipreading from non-frontal views (poses). In particular, we have focused on the use of profile views, and proposed two approaches for lipreading on basis of visual features extracted from such views: (a) Direct statistical modeling of the features, namely use of view-dependent statistical models; and (b) Normalization of such features by their projection onto the “space ” of frontal-view visual features, which allows employing one set of statistical models for all available views. The latter approach has been considered for two only poses (frontal and profile views), and for visual features of a specific dimensionality. In this paper, we further extend this work, by investigating its applicability to the case where data from three views are available (frontal, left- and right-profile). In addition, we examine the effect of visual feature dimensionality on the pose-normalization approach. Our experiments demonstrate that results generalize well to three views, but also that feature dimensionality is crucial to the effectiveness of the approach. In particular, feature dimensionality larger than 30 is detrimental to multi-pose visual speech recognition performance. Index Terms: Audio-visual automatic speech recognition (AVASR), pose invariance, profile and frontal views, lipreading
Keyphrases
extended pose-invariant lipreading system visual feature profile view feature dimensionality specific dimensionality multi-pose visual speech recognition performance frontal view view-dependent statistical model visual feature dimensionality pose-normalization approach recent work index term audio-visual automatic speech recognition frontal-view visual feature direct statistical modeling available view latter approach statistical model pose invariance non-frontal view
