DMCA
A Markov random field model for term dependencies
Cached
Download Links
| Citations: | 288 - 55 self |
BibTeX
@MISC{Metzler_amarkov,
author = {Donald Metzler and W. Bruce Croft},
title = {A Markov random field model for term dependencies},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
This paper develops a general, formal framework for modeling term dependencies via Markov random fields. The model allows for arbitrary text features to be incorporated as evidence. In particular, we make use of features based on occurrences of single terms, ordered phrases, and unordered phrases. We explore full independence, sequential dependence, and full dependence variants of the model. A novel approach is developed to train the model that directly maximizes the mean average precision rather than maximizing the likelihood of the training data. Ad hoc retrieval experiments are presented on several newswire and web collections, including the GOV2 collection used at the TREC 2004 Terabyte Track. The results show significant improvements are possible by modeling dependencies, especially on the larger web collections.
Keyphrases
term dependency markov random field model web collection several newswire formal framework terabyte track gov2 collection markov random field training data full dependence variant sequential dependence ad hoc retrieval experiment significant improvement mean average precision full independence arbitrary text feature single term unordered phrase novel approach
