Eurospeech 2001- Scandinavia Word Level Confidence Annotation using Combinations of Features
Abstract:
This paper describes the development of a word-level confidence metric suitable for use in a dialog system. Two aspects of the problems are investigated: the identification of useful features and the selection of an effective classifier. We find that two parse-level features, Parsing-Mode and Slot-Backoff-Mode, provide annotation accuracy comparable to that observed for decoder-level features. However, both decoderlevel and parse-level features independently contribute to confidence annotation accuracy. In comparing different classification techniques, we found that Support Vector Machines (SVMs) appear to provide the best accuracy. Overall we achieve 39.7 % reduction in annotation uncertainty for a binary confidence decision in a travel-planning domain. 1.
Citations
| 4514 | Statistical Learning Theory – Vapnik - 1998 |
| 1103 | A Tutorial on Support Vector Machines for Pattern Recognition – Burges - 1998 |
| 31 | Cono/dence Measures for Spontaneous Speech Recognition – Schaaf, Kemp - 1997 |
| 29 | Using Word Probabilities as Confidence Measures – Wessel, Macherey, et al. - 1998 |
| 28 | Error-responsive feedback mechanisms for speech recognition – Chase - 1997 |
| 19 | Large Vocabulary Speaker-independent Continuous Speech Recognition: The Sphinx System – Lee - 1988 |
| 8 | New Features for Confidence Annotation – Bansal, Ravishankar - 1998 |
| 4 | Beaufays et al, \Neural - network based measure of con for word recognition – Weintraub, Beaufays - 1997 |
| 4 | Confidence Measures for Dialogue Management – San-Segun, Pellom, et al. - 2000 |
| 1 | Eric Thayer, et al., “Creating Natural – Rudnicky - 1999 |
| 1 | Vassilios Digalakis and Patti Price, “Combining Knowledge Sources to Recorder N-Best Speech Hypothesis Lists – Rayner, Carter - 1994 |
| 1 | Ravishankar implemented this feature – Bansal, K - 1998 |
