DMCA
Using multiple query aspects to build test collections without human relevance judgments (2009)
Cached
Download Links
| Venue: | In: Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval |
| Citations: | 6 - 0 self |
BibTeX
@INPROCEEDINGS{Efron09usingmultiple,
author = {Miles Efron},
title = {Using multiple query aspects to build test collections without human relevance judgments},
booktitle = {In: Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval},
year = {2009}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract. Collecting relevance judgments (qrels) is an especially challenging part of building an information retrieval test collection. This paper presents a novel method for creating test collections by offering a substitute for relevance judgments. Our method is based on an old idea in IR: a single information need can be represented by many query articulations. We call different articulations of a particular need query aspects. By combining the top k documents retrieved by a single system for multiple query aspects, we build judgment-free qrels whose rank ordering of IR systems correlates highly with rankings based on human relevance judgments. 1
Keyphrases
human relevance judgment multiple query aspect test collection relevance judgment different articulation single system top document rank ordering information retrieval test collection old idea judgment-free qrels challenging part single information need many query articulation ir system novel method particular need query aspect
