Traditional image retrieval methods require a “query image” to initiate a search for members of an image category. However, when the image database is unstructured, and when the category is semantic and resides only in the mind of the user, there is no obvious way to begin (the “page zero ” problem). We propose a new mathematical framework for relevance feedback based on mental matching and starting from a random sample of images. At each iteration the user declares which of several displayed images is closest to his category; performance is measured by the number of iterations necessary to display an instance. Our core contribution is a Bayesian formulation which scales to large databases with no semantic annotation. The two key components are a response model which accounts for the user’s subjective perception of similarity and a display algorithm which seeks to maximize the flow of information. Experiments with real users and a database with 20,000 images demonstrate the efficiency of the search process. 1.

The Mental Image Search paradigm allows the user to retrieve images which match the target image s/he has in mind without a starting example. We present a novel approach for this paradigm which enables multiple descriptor range-query, which is necessary to match the more or less precise idea of the user's mental image. In a simple and intuitive way, sophisticated queries can be formulated on the visual appearance of the mental image components.

We present a novel technique for processing image similarity search by using an approach that takes inspiration from text retrieval techniques. In our approach images are indexed by using visual terms taken from a visual lexicon obtained clustering regions of images in the dataset. A weighting and matching schema is defined that allow effective image retrieval to be performed by using inverted files, thus requiring reduced storage space and achieving high efficiency. 1

c t i v it y e p o r t 2009 Table of contents

The spatial arrangement of visual elements of an image, i.e. the visual composition, is a research subject in the domain of visual arts which include painting, film, etc. Film experts face the problem of retrieval of visual compositions in film on a daily basis. Although, visual composition is a crucial element to consider in content-based video retrieval, little scientific effort has been invested into this problem so far. Actually, it is unclear if content-based retrieval of visual compositions is feasible. We present a user study conducted to investigate the feasibility of content-based retrieval of visual compositions as they are understood by film experts. For that reason, we create a data set derived from real world material and let the film experts evaluate the retrieval performance. The user study investigates the applicability of state-of-the-art visual features and shows differences in evaluations by film experts (test group) and computer scientists (reference group). 1.