User-defined classes in large generalist image databases are often composed of several groups of images and span very di#erent scales in the space of low-level visual descriptors. The interactive retrieval of such image classes is then very di#cult. To address this challenge, we propose and evaluate here two general improvements of SVM-based relevance feedback methods. First, to optimize the transfer of information between the user and the system, we focus on the criterion employed by the system for selecting the images presented to the user at every feedback round. We put forward a new active learning selection criterion that minimizes redundancy between the candidate images shown to the user. Second, for image classes having very di#erent scales, we find that a high sensitivity of the SVM to the scale of the data brings about a low retrieval performance. We then argue that insensitivity to scale is desirable in this context and we show how to obtain it by the use of specific kernel functions. The experimental evaluation of both ranking and classification performance on several image databases confirms the e#ectiveness of our selection criterion and of the use of kernels that reduce the sensitivity of SVMs to the scale of the data.

Introduction The di#culty and cost of providing rich and reliable textual annotations for images in large databases, as well as the "linguistic gap" associated to these annotations, explains why the retrieval of images based directly on their visual content (content-based image retrieval, CBIR) is of high interest today [16]. In the early years of research in CBIR, the focus was on query by visual example (QBVE): a search session begins by presenting an example image (or sketch) to the search engine as a visual query, then the engine returns images that are visually similar to the query image. More recently, the concept of semantic gap has been extensively used in the CBIR research community to express the discrepancy between the low-level features that can be readily extracted from the images and the descriptions that are meaningful for the users. The automatic association of such descriptions to the low-level features is currently only feasible for very restricted domains and appl

Relevance feedback approaches based on support vector machine (SVM) learning have been applied to significantly improve retrieval performance in content-based image retrieval (CBIR). Those approaches require the use of fixedlength image representations because SVM kernels represent an inner product in a feature space that is a non-linear transformation of the input space. Many region-based CBIR approaches create a variable length image representation and define a similarity measure between two variable length representations. The standard SVM approach cannot be applied to this approach because it violates the requirements that SVM places on the kernel. Fortunately, a generalized SVM (GSVM) has been developed that allows the use of an arbitrary kernel. In this paper, we present an initial investigation into utilizing a GSVM-based relevance feedback learning algorithm. Since GSVM does not place restrictions on the kernel, any image similarity measure can be used. In particular, the proposed approach uses an image similarity measure developed for region-based, variable length representations. Experimental results over real world images demonstrate the efficacy of the proposed method.

Abstract not found

This paper presents a generalized Bayesian framework for relevance feedback in content-based image retrieval. The proposed feedback technique is based on Bayesian learning method and incorporates a time-varying user model into the formulation. We define the user model with two terms: a target query and a user conception. The target query is aimed to learn the common features from relevant images so as to specify the user’s ideal query. The user conception is aimed to learn a parameter set to determine the time-varying matching criterion. Therefore, at each feedback step, the learning process updates not only the target distribution but also the target query and the matching criterion. In addition, another objective of this paper is to conduct the relevance feedback on images represented in region level. We formulate the matching criterion using a weighting scheme and proposed a region clustering technique to determine the region correspondence between relevant images. With the proposed region clustering technique, we derive a representation in region level to characterize the target query. Experiments demonstrate that the proposed method combined with time-varying user model indeed achieves satisfactory results and our proposed region-based techniques further improve the retrieval accuracy. Index terms – Content-based image retrieval, relevance feedback, Bayesian learning, target query, user conception, region clustering, region correspondence. I.

the input data is a collection of images that are annotated with a given keyword, such as “car”. The problem is to attribute the annotation to specific parts of the images. There exists plenty of suitable input data readily

This thesis deals with the problem of £nding images that contain a given query sub-image, the so-called Content-Based sub-Image Retrieval (CBsIR) problem. We propose a scheme named the Hierarchical Tree Matching (HTM), which relies on a hierarchical tree that encodes the color features of image tiles stored in turn as an index sequence. The index sequences of both candidate images and the query sub-image are then compared using a search strategy based on the hierarchical tree structure in order to rank the database images with respect to the query. Our experimental results on a database of over 10,000 images and disk-resident metadata suggest that the HTM scheme can be very effective and ef£cient and performs much better than an alternative method in retrieving the original images, i.e., the ones from which the query sub-images are extracted. To further improve the quality of retrieval, we also investigate the use of feedback to better capture the user’s intention. The user can thus provide feedback on the retrieved results by identifying images of his/her interest. Combined with the HTM strategy, we use a relevance feedback approach based on a tile re-weighting scheme. Our experiments show that this learning approach is quite effective, improving the retrieval within very few iterations.

Probabilistic feature relevance learning (PFRL) is an effective method for adaptively computing local feature relevance in content-based image retrieval. It computes flexible retrieval metrics for producing neighborhoods that are elongated along less relevant feature dimensions and constricted along most influential ones. Based on the observation that regions in an image have unequal importance for computing image similarity, we propose a probabilistic method inspired by PFRL, probabilistic region relevance learning (PRRL), for automatically estimating region relevance based on user's feedback. PRRL can be used to set region weights in region-based image retrieval frameworks that use an overall image-to-image similarity measure. Experimental results on general-purpose images show the effectiveness of PRRL in learning the relative importance of regions in an image.

image retrieval with high-level semantics using decision tree learning