In the QBIC (Query By Image Content) project we are studying methods to query large on-line image databases using the images' content as the basis of the queries. Examples of the content we use include color, texture, and shape of image objects and regions. Potential applications include medical ("Give me other images that contain a tumor with a texture like this one"), photo-journalism ("Give me images that have blue at the top and red at the bottom"), and many others in art, fashion, cataloging, retailing, and industry. We describe a set of novel features and similarity measures allowing query by color, texture, and shape of image object. We demonstrate the effectiveness of the QBIC system with normalized precision and recall experiments on test databases containing over 1000 images and 1000 objects populated from commercially available photo clip art images, and of images of airplane silhouettes. We also consider the efficient indexing of these features, specifically addre...

. We introduce a very fast and reliable method for shape similarity retrieval in large image databases which is robust with respect to noise, scale and orientation changes of the objects. The maxima of curvature zero crossing contours of Curvature Scale Space (CSS) image are used to represent the shapes of object boundary contours. While a complex boundary is represented by about five pairs of integer values, an effective indexing method based on the aspect ratio of the CSS image, eccentricity and circularity is used to narrow down the range of searching. Since the matching algorithm has been designed to use global information, it is sensitive to major occlusion, but some minor occlusion will not cause any problems. We have tested and evaluated our method on a prototype database of 450 images of marine animals with a vast variety of shapes with very good results. The method can either be used in real applications or produce a reliable shape description for more complicated images when ...

There are essential differences between multimedia databases (which may contain complicated objects, such as images), and traditional databases. These differences lead to interesting new issues, and in particular cause us to consider new types of queries. For example, in a multimedia database it is reasonable and natural to ask for images that are somehow "similar to" some fixed image. Furthermore, there are different ways of obtaining and accessing information in a multimedia database than information in a traditional database. For example, in a multimedia database, it might be reasonable to have a query that asks for, say, the top 10 images that are similar to a fixed image. This is in contrast to a relational database, where the answer to a query is simply a set. (Of course, in a relational database, the result to a query may be sorted in some way for convenience in presentation, such as sorting department members by salary, but logically speaking, the result is still simply a set, ...

We study how to measure the degree of similarity between two image contours. We propose an approach for comparing contours that takes into account deformations in object shape, the articulation of parts, and variations in the shape and size of portions of objects. Our method uses dynamic programming to compute the minimum cost of bringing one shape into the other via local deformations. Using this as a starting point, we investigate the properties that such a cost function should have to model human performance and to perform usefully in a computer vision system. We suggest novel conditions on this cost function that help capture the part-based nature of objects without requiring any explicit decomposition of shapes into their parts. We then suggest several possible cost functions based on different physical models of contours, and describe experiments with these costs. 1 Introduction Detecting similarity is a key tool in interpretating images. In this paper we develop a measure of s...

What does it mean for a deforming object to be "moving" (see Fig. 1)? How can we separate the overall motion (a finite-dimensional group action) from the more general deformation (a di#eomorphism)? In this paper we propose a definition of motion for a deforming object and introduce a notion of "shape average" as the entity that separates the motion from the deformation. Our definition allows us to derive novel and e#cient algorithms to register non-equivalent shapes using region-based methods, and to simultaneously approximate and register structures in grey-scale images. We also extend the notion of shape average to that of a "moving average" in order to track moving and deforming objects through time.

Abstract. We study some Riemannian metrics on the space of smooth regular curves in the plane, viewed as the orbit space of maps from S1 to the plane modulo the group of diffeomorphisms of S1, acting as reparameterizations. In particular we investigate the metric for a constant A> 0:

Advanced perceptual systems are faced with the problem of securing a principled relationship between the world and its internal representation. I propose a unified approach to visual representation, based on Shepard's (1968) notion of second-order isomorphism. According to the proposed theory, a shape is represented internally by the responses of a few tuned modules, each of which is broadly selective for some reference shape, whose similarity to the stimulus it measures. The result is a philosophically appealing, computationally feasible, biologically credible, and formally veridical representation of a distal shape space. This approach supports representation of and discrimination among shapes radically different from the reference ones, while bypassing the need for the computationally problematic decomposition into parts; it also addresses the needs of shape categorization, and can be used to derive a range of models of perceived similarity. Representation is Representation of Sim...

We describe a multi-feature hierarchical algorithm to efficiently match N objects (templates) with an image using distancetransforms (DTs). The matching is under translation, but it can cover more general transformations by generating the various transformed templates explicitly. The novel part of the algorithm is that, in addition to acoarse-to-fine search over the translation parameters, the N templates aregrouped off-line into a template hierarchy based on their similarity. This way, multiple templates can be matched resulting in various speed-up factors. Furthermore, in matching, features are distinguishedby type and separate DT's arecomputed for each type (e.g. basedonedge orientations). These concepts are illustrated in the application of traffic sign detection.

The study of 2D shapes and their similarities is a central problem in the field of vision. It arises in particular from the task of classifying and recognizing objects from their observed silhouette. Defining natural distances between 2D shapes creates a metric space of shapes, whose mathematical structure is inherently relevant to the classification task. One intriguing metric space comes from using conformal mappings of 2D shapes into each other, via the theory of Teichmüller spaces. In this space every simple closed curve in the plane (a “shape”) is represented by a ‘fingerprint ’ which is a diffeomorphism of the unit circle to itself (a differentiable and invertible, periodic function). More precisely, every shape defines to a unique equivalence class of such diffeomorphisms up to right multiplication by aMöbius map. The fingerprint does not change if the shape is varied by translations and scaling and any such equivalence class comes from some shape. This coset space, equipped with the infinitesimal Weil-Petersson (WP) Riemannian norm is a metric space. In this space, it appears very likely to be true that the shortest path between each two shapes is unique, and is given by a geodesic connecting them. Their distance from each other is given by integrating the WP-norm along that geodesic. In this paper we concentrate on solving the “welding ” problem of “sewing” together conformally the interior and exterior of the unit circle, glued on the unit circle by a given diffeomorphism, to obtain the unique 2D shape associated with this diffeomorphism. This will allow us to go back and forth between 2D shapes and their representing diffeomorphisms in this “space of shapes”. 1

We describe a method for shape-based image database search that uses deformable prototypes to represent categories. Rather than directly comparing a candidate shape with all shape entries in the database, shapes are compared in terms of the types of nonrigid deformations (differences) that relate them to a small subset of representative prototypes. To solve the shape correspondence and alignment problem, we employ the technique of modal matching, an information-preserving shape decomposition for matching, describing, and comparing shapes despite sensor variations and nonrigid deformations. In modal matching, shape is decomposed into an ordered basis of orthogonal principal components. We demonstrate the utility of this approach for shape comparison in 2-D image databases.