We have witnessed great interest and a wealth of promise in content-based image retrieval as an emerging technology. While the last decade laid foundation to such promise, it also paved the way for a large number of new techniques and systems, got many new people involved, and triggered stronger association of weakly related fields. In this article, we survey almost 300 key theoretical and empirical contributions in the current decade related to image retrieval and automatic image annotation, and in the process discuss the spawning of related subfields. We also discuss significant challenges involved in the adaptation of existing image retrieval techniques to build systems that can be useful in the real world. In retrospect of what has been achieved so far, we also conjecture what the future may hold for image retrieval research.

The last decade has witnessed great interest in research on content-based image retrieval. This has paved the way for a large number of new techniques and systems, and a growing interest in associated fields to support such systems. Likewise, digital imagery has expanded its horizon in many directions, resulting in an explosion in the volume of image data required to be organized. In this paper, we discuss some of the key contributions in the current decade related to image retrieval and automated image annotation, spanning 120 references. We also discuss some of the key challenges involved in the adaptation of existing image retrieval techniques to build useful systems that can handle real-world data. We conclude with a study on the trends in volume and impact of publications in the field with respect to venues/journals and sub-topics.

Advances in data storage and image acquisition technologies have enabled the creation of large image datasets. In this scenario, it is necessary to develop appropriate information systems to efficiently manage these collections. The commonest approaches use the so-called Content-Based Image Retrieval (CBIR) systems. Basically, these systems try to retrieve images similar to a user-defined specification or pattern (e.g., shape sketch, image example). Their goal is to support image retrieval based on content properties (e.g., shape, color, texture), usually encoded into feature vectors. One of the main advantages of the CBIR approach is the possibility of an automatic retrieval process, instead of the traditional keyword-based approach, which usually requires very laborious and time-consuming previous annotation of database images. The CBIR technology has been used in several applications such as fingerprint identification, biodiversity information systems, digital libraries, crime prevention, medicine, historical research, among others. This paper aims to introduce the problems and challenges concerned with the creation of CBIR systems, to describe the existing solutions and applications, and to present the state of the art of the existing research in this area.

Abstract. In this demo proposal, we describe our prototype application, SIERRA, which combines text-based and content-based image retrieval and allows users to link together image content of varying document granularity with related data like annotations. To achieve this, we use the concept of superimposed information (SI), which enables users to (a) deal with information of varying granularity (sub-document to complete document), and (b) select or work with information elements at sub-document level while retaining the original context. Description In many image-based applications, like biomedical teaching, research, and diagnosis, there is need to link (or integrate) image content with other multimedia information: text annotations, metadata (keywords or ontological terms), audio-visual presentations, etc. Not only does this contribute to richer image descriptions, it also helps in more effective retrieval of images and related information [10]. Further, for complex images (e.g., images with plenty of detail, or with specific hard-to find details), there

Abstract—This work presents an image representation and matching framework for image categorization in medical image archives. Categorization enables to determine automatically, based on the image content, the examined body region and imaging modality. It is a basic step in content-based image retrieval (CBIR) systems, the goal of which is to augment textbased search with visual information analysis. CBIR systems are currently being integrated with Picture-Archiving and Communication Systems (PACS) for increasing the overall search capabilities and tools available to radiologists. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (GMM) along with information-theoretic image matching via the Kullback Leibler (KL) measure. The GMM-KL framework is used for matching and categorizing x-ray images by body regions. A multi-dimensional feature space is used to represent the image input, including intensity, texture and spatial information. Unsupervised clustering via the GMM is used to extract coherent regions in feature space which are then used in the matching process. A dominant characteristic of the radiological images is their poor contrast and large intensity variations. This presents a challenge to matching between images and is handled via an illumination invariant representation. The GMM-KL framework is evaluated for image categorization and image retrieval on a dataset of 1500 radiological images. A classification rate of 97.5% was achieved. The classification results compare favorably with reported global and local representation schemes. Precision vs. Recall curves indicate a strong retrieval result as compared with

Content--based image retrieval systems (CBIRSs) have frequently been proposed for the use in medical image databases and PACS. Still, only few systems were developed and used in a real clinical environment. It rather seems that medical professionals define their needs and computer scientists develop systems based on data sets they receive with little or no interaction between the two groups. A first study on the diagnostic use of medical image retrieval also shows an improvement in diagnostics when using CBIRSs which underlines the potential importance of this technique.

Abstract. Future digital libraries will not only contain pure text documents, but increasingly hold massive amounts of compound documents which comprise many multimedia objects, e.g. texts, images, audio, and video. Already existing collections of documents, e.g. all electronic health records of one clinic can form a digital library with millions of multimedia objects and a total storage of several terabytes. It is therefore important to provide ways for effective and efficient retrieval for those collections. This paper proposes a novel approach for compound document matching using a filter-and-refinement algorithm for similarity-based retrieval. At the same time, this approach increases the effectiveness by establishing only semantically meaningful matches and providing greater expressiveness in queries by restricting the number of allowed matches to a single query object. 1

A Content-Based Approach to Medical Image Database Retrieval Content-based image retrieval (CBIR) makes use of image features, such as color and texture, to index images with minimal human intervention. Content-based image retrieval can be used to locate medical images in large databases. This chapter introduces a content-based approach to medical image retrieval. Fundamentals of the key components of content-based image retrieval systems are introduced first to give an overview of this area. A case study, which describes the methodology of a CBIR system for retrieving digital mammogram database, is then presented. This chapter is intended to disseminate the knowledge of the CBIR approach to the applications of medical image management and to attract greater interest from various research communities to rapidly advance research in this field. 1.

In this paper, we introduce a new approach for content-based similarity search for brain images. Based on the keyblock representation, our framework employs the Principal Component Analysis to reduce the dimensionality and improve the computational efficiency. Moreover, the “similarity ” between two images is measured using both the Histogram Model and the Summed Euclidean Distance. We performed experiments on different fMRI datasets, and compared the proposed framework with the keyblock approach. The results of the experiments demonstrated the improved effectiveness and efficiency of the proposed approach in similarity searches. Index Terms — Biomedical imaging, Vector quantization, Information retrieval

Visual inspection of cells is a fundamental tool for discovery in biological science. Modern robotic microscopes are able to capture thousands of images from massively parallel experiments such as RNA interference (RNAi) or small-molecule screens. Such screens also benefit from lab automation, making large screens, e.g., genome-scale knockdown experiments, more feasible and common. As such, the bottleneck in large, imagebased screens has shifted to visual inspection and scoring by experts. In this paper, we describe the methods we have developed for automatic image cytometry. The paper demonstrates illumination normalization, foreground/background separation, cell segmentation, and shows the benefits of using a large number of individual cell measurements when exploring data from highthroughput screens.