• Documents
  • Authors
  • Tables
  • Log in
  • Sign up
  • MetaCart
  • DMCA
  • Donate

CiteSeerX logo

Advanced Search Include Citations
Advanced Search Include Citations

DMCA

A Gaussian prior for smoothing maximum entropy models (1999)

Cached

  • Download as a PDF

Download Links

  • [www.dtic.mil]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]
  • [reports-archive.adm.cs.cmu.edu]

  • Save to List
  • Add to Collection
  • Correct Errors
  • Monitor Changes
by Stanley F. Chen , Ronald Rosenfeld
Citations:253 - 2 self
  • Summary
  • Citations
  • Active Bibliography
  • Co-citation
  • Clustered Documents
  • Version History

BibTeX

@TECHREPORT{Chen99agaussian,
    author = {Stanley F. Chen and Ronald Rosenfeld},
    title = {A Gaussian prior for smoothing maximum entropy models},
    institution = {},
    year = {1999}
}

Share

Facebook Twitter Reddit Bibsonomy

OpenURL

 

Abstract

In certain contexts, maximum entropy (ME) modeling can be viewed as maximum likelihood train-ing for exponential models, and like other maximum likelihood methods is prone to overfitting of training data. Several smoothing methods for maximum entropy models have been proposed to address this problem, but previous results do not make it clear how these smoothing methods com-pare with smoothing methods for other types of related models. In this work, we survey previous work in maximum entropy smoothing and compare the performance of several of these algorithms with conventional techniques for smoothing n-gram language models. Because of the mature body of research in n-gram model smoothing and the close connection between maximum entropy and conventional n-gram models, this domain is well-suited to gauge the performance of maximum entropy smoothing methods. Over a large number of data sets, we find that an ME smoothing method proposed to us by Lafferty [1] performs as well as or better than all other algorithms under consideration. This general and efficient method involves using a Gaussian prior on the parame-ters of the model and selecting maximum a posteriori instead of maximum likelihood parameter values. We contrast this method with previous n-gram smoothing methods to explain its superior performance.

Keyphrases

maximum entropy model    gaussian prior    maximum entropy    conventional technique    n-gram language model    large number    maximum entropy smoothing    maximum likelihood parameter value    previous result    efficient method    exponential model    superior performance    survey previous work    n-gram model smoothing    maximum likelihood method    conventional n-gram model    mature body    close connection    maximum likelihood train-ing    certain context    related model    data set   

Powered by: Apache Solr
  • About CiteSeerX
  • Submit and Index Documents
  • Privacy Policy
  • Help
  • Data
  • Source
  • Contact Us

Developed at and hosted by The College of Information Sciences and Technology

© 2007-2019 The Pennsylvania State University