J. Wu and S. Khudanpur: "Combining Non-local, Syntactic and N-gram dependencies in Language Modeling," Proceedings of NLDB99, to appear, Klagenfurt, Austria, 1999  Home/Search   Document Details and Download   Summary   Related Articles   Check

....about the current partial parse T i . 9 So h Gamma3 might be useful, or at least t Gamma3 : The information extracted from T i might be made even more comprehensive if we took advantage of the maximum entropy estimation paradigm [2] We have had some success with such an approach already [13]. 10. PRELIMINARY RESULTS We have tested the SLM on the Wall Street Journal and Switchboard tasks [5] 12] Compared to the state of the art trigram language model, the SLM has a lower perplexity by 15 and 5 , respectively. It lowers the recognition error rate (WER) by 1 and 1 absolute, ....

J. Wu and S. Khudanpur: "Combining Non-local, Syntactic and N-gram dependencies in Language Modeling," Proceedings of NLDB99, to appear, Klagenfurt, Austria, 1999

....over N gram models have been proposed. Chelba and Jelinek [1, 2] have used a left to right parser to extract syntactic heads and used them to enhance trigram models. We have used a similar model which integrates N grams, headword statistics and topic dependencies in a maximum entropy framework [6] with considerable success. Consider the following sentence which illustrates the mechanism used by these models. Financial officials in the former British colony consider the contract essential to the revival of the Hong Kong futures exchange. The word consider is clearly more at home as part ....

.... form X w i Gamma2 ;w i Gamma1 ;nt i Gamma2 ;nt i Gamma1 P (w i Gamma1 ; w i Gamma2 ; nt i Gamma2 ; nt i Gamma1 ; w i jh i Gamma2 ; h i Gamma1 ) #[h i Gamma2 ; h i Gamma1 ; w i ] #[h i Gamma2 ; h i Gamma1 ] 6) on the frequency of headword N grams, then we get the ME model described in [6], which has the form P (w i jw i Gamma1 ; w i Gamma2 ; h i Gamma1 ; h i Gamma2 ) 1 Z Delta e (w i ) Delta e (w i Gamma1 ;w i ) Delta e (w i Gamma2 ;w i Gamma1 ;w i ) Deltae (h i Gamma1 ;w i ) Delta e (h i Gamma2 ;h i Gamma1 ;w i ) 7) If we remove the headword N gram ....

[Article contains additional citation context not shown here]

J. Wu and S. Khudanpur, "Combining Nonlocal, Syntactic and N-Gram Dependencies in Language Modeling ", Proceedings of Eurospeech'99, vol 5, pp2179-2182, September 6-10, 1999, Budapest, Hungary.

....[1] and its topic [8, 5] have been shown to improve the performance of statistical language models. It is natural to build a language model that uses all these sources of information. Maximum entropy (ME) is a technique for combining different sources of dependencies in one unified framework [7]. An ME model incorporating syntactic head word constraints, non terminal label constraints, and topic constraints with conventional N gram constraints has the form: P (wjw i Gamma1 ; w Gamma2 ; h Gamma1 ; h Gamma2 ; nt Gamma1 ; nt Gamma2 ; topic) 1) 1 Z Delta e (w) Delta e (w ....

....topic) is the normalization factor. w) is independent of histories and is thus called a marginal constraint. w Gamma1 ; w) topic; w) are history dependent and are called conditional constraints. Details of syntactic language modeling and topic sensitive language modeling are available in [1, 5, 7]. A fundamental difficulty in implementing such a complicated language model is the heavy computational cost of the training procedure. In state of the art training algorithms [3] the computational complexity for each iteration in the training is O(jXj Delta j Cj) where jXj is the number of ....

J. Wu and S. Khudanpur, "Combining Nonlocal, Syntactic and N-Gram Dependencies in Language Modeling ", Proceedings of Eurospeech'99, vol 5, pp21792182, Sep. 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC