DMCA
Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability. (2011)
Cached
Download Links
| Venue: | for Computational Linguistics. |
| Citations: | 124 - 15 self |
BibTeX
@INPROCEEDINGS{Clark11betterhypothesis,
author = {Jonathan H Clark and Chris Dyer and Alon Lavie and Noah A Smith},
title = {Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability.},
booktitle = {for Computational Linguistics.},
year = {2011},
pages = {176--181},
publisher = {Association}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract In statistical machine translation, a researcher seeks to determine whether some innovation (e.g., a new feature, model, or inference algorithm) improves translation quality in comparison to a baseline system. To answer this question, he runs an experiment to evaluate the behavior of the two systems on held-out data. In this paper, we consider how to make such experiments more statistically reliable. We provide a systematic analysis of the effects of optimizer instability-an extraneous variable that is seldom controlled for-on experimental outcomes, and make recommendations for reporting results more accurately.
