In current systems for browsing image collections, users are presented with sets of thumbnail images arranged in some default order on the screen. We are investigating whether it benefits users to have sets of thumbnails arranged according to their mutual similarity, so images that are alike are placed together. There are, of course, many possible definitions of "similarity": so far we have explored measurements based on low-level visual features, and on the textual captions assigned to the images. Here we describe two experiments, both involving designers as the participants, examining whether similarity-based arrangements of the candidate images are helpful for a picture selection task. Firstly, the two types of similarity-based arrangement were informally compared. Then, an arrangement based on visual similarity was more formally compared with a control of a random arrangement. We believe this work should be of interest to anyone designing a system that involves presenting sets of images to users. Keywords Image retrieval, information visualisation, evaluation.

1988, no part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying or otherwise without the prior written permission of the publisher.

We report on the progressive investigation of techniques for supporting interactive query reformulation in the TREC Interactive Track task. Two major issues were explored over four successive years: various methods of term suggestion; and, interface design to support different system functionalities. Each year’s results led to the following year’s investigation, with respect to both of these major issues. This paper presents first the general motivation for the entire series of studies; then an overview of each year’s investigation, its results, and how they influenced the next year’s investigation. We discuss what we believe has been learned through this series of investigations about effective term suggestion, usable and useful interface design, and the relationships between these two in support of the TREC Interactive Track task. We conclude with some comments about the general methodology which we employed over this series of studies, and its relevance to the development and evaluation of interactive information retrieval systems in general.

The Ostensive Model proposes a manner of structuring the uncertainty associated with individual relevance judgements as sources of evidence in relevance feedback. It proposes temporal profiles of uncertainty, motivating the application of a particular class of discount function with respect to the age of the evidence. This paper presents an initial evaluation of the relative effectiveness of different uncertainty discount functions. A novel direct manipulation interface to a multimedia retrieval system embodying the Ostensive Model is outlined briefly. The paper describes the construction and characteristics of a new image test collection utilising multiple binary relevance assessments. The use of such multiple assessments and multiple interpretations of them are discussed. The evaluation environment is detailed in terms of the interface, test collection, and tasks set to users. Multiple interpretations of the results, and the statistical significance of comparisons are presented. Th...

In previous work, the Sonic Browser was used for browsing large data sets of music [1]. In this paper, we report results from an updated version of the Sonic Browser for managing general sound resources on personal computers. In particular, we have evaluated browsing of everyday sounds. The investigation was directed at comparing browsing single versus multiple stream audio. The problem of sound resource browsing for multimedia designers is the specific area of focus for our experiment. Finally, we conclude with current trends of our research for further improvement of the system. 1.

The aim of an information retrieval system is to find relevant documents, thus relevance is a (if not `the') central concept of information retrieval. Notwithstanding its importance, and the huge amounts of research on this topic in the past, relevance is not yet a well understood concept, also because of an inconsistently used terminology. In this paper, I try to clarify this issue, classifying the various kinds of relevance. I show that: (i) there are many kinds of relevance, not just one; (ii) these kinds can be classified in a formally defined four dimensional space; and (iii) such classification helps us to understand the nature of relevance and relevance judgement. Finally, the consequences of this classification on the design and evaluation of information retrieval systems are analysed.

A growing body of studies is developing approaches to evaluating human interaction with Web search engines, including the usability and effectiveness of Web search tools. This study explores a user-centered approach to the evaluation of the Web search engine Inquirus – a Web meta-search tool developed by researchers from the NEC Research Institute. The goal of the study reported in this paper was to develop a user-centered approach to the evaluation including: (1) effectiveness: based on the impact of users ' interactions on their information problem and information seeking stage, and (2) usability: including screen layout and system capabilities for users. Twenty-two (22) volunteers searched Inquirus on their own personal information topics. Data analyzed included: (1) user pre- and post-search questionnaires and (2) Inquirus search transaction logs. Key findings include: (1) Inquirus was rated highly by users on various usability measures, (2) all users experienced some level of shift/change in their information problem, information seeking, and personal knowledge due to their Inquirus interaction, (3) different users experienced different levels of change/shift, and (4) the search measure precision did not correlate with other user-based measures. Some users experienced major changes/shifts in various user-based variables, such as information problem or information seeking stage with a search of low precision and vice versa. Implications for the development of user-centered approaches to the evaluation of Web and IR systems and further research are discussed. 2

Digital libraries (DLs) are complex information systems and therefore demand formal foundations lest development e#orts diverge and interoperability su#ers. In this paper, we propose the fundamental abstractions of Streams, Structures, Spaces, Scenarios, and Societies (5S), which contribute to define digital libraries rigorously and usefully. Streams are sequences of abstract items used to describe static and dynamic content. Structures can be defined as labeled directed graphs, which impose organization. Spaces are sets of abstract items and operations on those sets that obey certain rules. Scenarios consist of sequences of events or actions that modify states of a computation in order to accomplish a functional requirement. Societies comprehend entities and the relationships between and among them. Together these abstractions relate and unify concepts, among others, of digital objects, metadata, collections, and services required to formalize and elucidate "digital libraries". The applicability, versatility and unifying power of the theory is demonstrated through its use in three distinct applications: building and interpretation of a DL taxonomy, analysis of case studies of digital libraries, and utilization as a formal basis for a DL description language. Keywords: digital libraries, theory, foundations, definitions, applications 1 1 Motivation Digital libraries are extremely complex information systems. The proper concept of a digital library seems hard to completely understand and evades definitional consensus. Di#erent views (e.g., historical, technological) and perspectives (e.g., from the library and information science, information retrieval, or human-computer interaction communities) have led to a myriad of di#ering definitions. Licklider, in his seminal ...

We report quantitative and qualitative results of an empirical evaluation to determine whether automated assistance improves searching performance and when searchers desire system intervention in the search process. Forty participants interacted with two fully functional information retrieval systems in a counterbalanced, within-participant study. The systems were identical in all respects except that one offered automated assistance and the other did not. The study used a client-side automated assistance application, an approximately 500,000-document Text REtrieval Conference content collection, and six topics. Results indicate that automated assistance can improve searching performance. However, the improvement is less dramatic than one might expect, with an approximately 20 % performance increase, as measured by the number of userselected relevant documents. Concerning patterns of interaction, we identified 1,879 occurrences of searcher– system interactions and classified them into 9 major categories and 27 subcategories or states. Results indicate that there are predictable patterns of times when searchers desire and implement searching assistance. The most common three-state pattern is Execute Query–View Results: With Scrolling–View Assistance. Searchers appear receptive to automated assistance; there is a 71 % implementation rate. There does not seem to be a correlation between the use of assistance and previous searching performance. We discuss the implications for the design of information retrieval systems and future research directions.

Although no unified definition of the concept of search strategy in Information Retrieval (IR) exists so far, its importance is manifest: nonexpert users, directly interacting with an IR system, apply alimited portfolio of simple actions; they do not know how to react in critical situations; and they often do not even realize that their difficulties are due to strategic problems. Auser interface to an IR system should therefore provide some strategic help, focusing user’s attention on strategic issues and providing tools to generate better strategies. Because neither the user nor the system can autonomously solve the information problem, but they complement each other, we propose acollaborative coaching approach,inwhichthetwopartnerscooperate:theuser retains the control of the session and the system provides suggestions. The effectiveness of the approach is demonstrated by a conceptual analysis, a prototype knowledge-based system named FIRE, and its evaluation through informal laboratory experiments.