Abstract not found

e passes. Two pairs of runs were submitted: in one pair queries remained constant, but in the other query terms were reweighted when fresh relevance information became available. VLC track Four runs on the full database were submitted, together with one each on the 10# and 1# collections. Unexpectedly, unexpanded queries did better than expanded ones; the best run used all topic #elds and some adjacent term pairs from the topics. The best expanded run used one of the TREC#7 ad hoc query sets #expanded on disks 1#5#. Interactive track Two pairwise comparisons were made: Okapi with relevance feedback against Okapi without, and Okapi without against ZPrise without. Okapi without performed somewhat worse than ZPrise, and Okapi with only partially recovered the de#cit. # Microsoft Research Ltd, 1 Guildhall Street, Cambridge CB2 3NH, UK, and City University, London, UK. email ser@microsoft.com y Microsoft Research Ltd, 1 Guildha

this paper; comparisons between passage and non-passage runs can be seen in a few of the tables.

... this report is concerned with the Adaptive Filtering track only. There is a separate report in this volume [1] on the Microsoft Research Cambridge participation in QA track. A number

Introduction: summary General City submitted two runs each for the automatic ad hoc, very large collection track, automatic routing and Chinese track; and took part in the interactive and #ltering tracks. There were no very signi#cant new developments; the same Okapi-style weighting as in TREC#3 and TREC#4 was used this time round, although there were attempts, in the ad hoc and more notably in the Chinese experiments, to extend the weighting to cover searches containing both words and phrases. All submitted runs except for the Chinese incorporated run-time passage determination and searching. The Okapi back-end search engine has been considerably speeded, and a few new functions incorporated. See Section 3. Automatic ad hoc #and VLC track# After a good deal of rather fruitless exploration, including further work with two-term phrases, the method #nally used was much the same as in TRECs 3 and 4: expansion using terms from the top documents retrieved by

Digital libraries consisting of huge amounts of multimedia data become increasingly common on the Internet. Disclosure of this multimedia content is very important for many applications. Unfortunately, multimedia query formulation in general is a difficult task and depends among other things on the context in which the user's request appears. This thesis investigates the role of knowledge in multimedia information retrieval and introduces the design and implementation of a exible, adaptable multimedia query processor in the Mirror retrieval model in an open distributed digital library environment that incorporates domain knowledge stored in concept thesauri and user knowledge in the form of relevance feedback. Multimedia information retrieval and cognitive science are closely related, especially in intelligent multimedia information retrieval. Therefore, some important theories in cognitive science and their parallels in computer perception, concept reasoning and combination will also ...

This paper is a personal take on the history of evaluation experiments in information retrieval. It describes some of the early experiments that were formative in our understanding, and goes on to discuss the current dominance of TREC (the Text REtrieval Conference) and to assess its impact. Keywords: information retrieval, evaluation, experimentation, relevance, Cranfield, TREC.

It would appear from all we have heard and read that Internet access to library catalogs is at best a mixed blessing. It will be difficult if not impossible for Internet access to improve online catalog use unless the catalogs are each

The paper highlights the ever increasing complexity in the evaluation of IR systems which has arisen over the last decade. Relevance, cognition, user behaviour, interaction and a changing view of the boundaries of the system are considered to be contributory factors. Issues such as laboratory versus operational systems, black-box versus diagnostic experiments, and qualitative and quantitative methods, are discussed and supported by examples drawn from three groups of evaluative experiments: weighted searching on a front end system, information seeking behaviour and the use of OPACs, and the OKAPI experimental retrieval system. The volume edited by Sparck Jones and published in 1981, Information Retrieval Experiment, remains the one substantial work on the evaluation of IR systems. The first chapter, by one of the present authors (Robertson, 1981), ended with the thought that the succeeding twenty years might see as much change in this field as the previous twenty: If, in 2001, this entire chapter is obsolete, so much the better! As we are now half way through that period, an overview is appropriate. It is the contention of the present paper that the field has changed substantially in ten years. 1