DMCA
Estimation of probabilities from sparse data for the language model component of a speech recognizer (1987)
Cached
Download Links
| Venue: | IEEE Transactions on Acoustics, Speech and Signal Processing |
| Citations: | 798 - 2 self |
BibTeX
@INPROCEEDINGS{Katz87estimationof,
author = {Slava M. Katz},
title = {Estimation of probabilities from sparse data for the language model component of a speech recognizer},
booktitle = {IEEE Transactions on Acoustics, Speech and Signal Processing},
year = {1987},
pages = {400--401}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract-The description of a novel type of rn-gram language model is given. The model offers, via a nonlinear recursive procedure, a com-putation and space efficient solution to the problem of estimating prob-abilities from sparse data. This solution compares favorably to other proposed methods. While the method has been developed for and suc-cessfully implemented in the IBM Real Time Speech Recognizers, its generality makes it applicable in other areas where the problem of es-timating probabilities from sparse data arises. Sparseness of data is an inherent property of any real text, and it is a problem that one always encounters while collecting fre-quency statistics on words and word sequences (m-grams) from a text of finite size. This means that even for a very large data col-lection, the maximum likelihood estimation method does not allow Turing’s estimate PT for a probability of a word (m-gram) which occurred in the sample r times is r* PT = where r We call a procedure of replacing a count r with a modified count r ’ “discounting ” and a ratio rt/r a discount coefficient dr. When r ’ = r*, we have Turing’s discounting. Let us denote the m-gram wl, *.., w, as wy and the number of times it occurred in the sample text as c(wT). Then the maxi-mum likelihood estimate is
Keyphrases
speech recognizer language model component word sequence real text ratio rt rn-gram language model fre-quency statistic es-timating probability estimate pt novel type nonlinear recursive procedure maximum likelihood estimation method model offer sample text inherent property sample time sparse data arises ibm real time speech recognizers space efficient solution large data col-lection modified count discounting discount coefficient dr sparse data maxi-mum likelihood estimate m-gram wl abstract-the description finite size
