Retrieving images from large and varied collections using image content as a key is a challenging and important problem. We present a new image representation which provides a transformation from the raw pixel data to a small set of image regions which are coherent in color and texture. This "Blobworld" representation is created by clustering pixels in a joint color-texture-position feature space. The segmentation algorithm is fully automatic and has been run on a collection of 10,000 natural images. We describe a system that uses the Blobworld representation to retrieve images from this collection. An important aspect of the system is that the user is allowed to view the internal representation of the submitted image and the query results. Similar systems do not offer the user this view into the workings of the system; consequently, query results from these systems can be inexplicable, despite the availability of knobs for adjusting the similarity metrics. By finding image regions whi...

Junctions are signi cant features in images with an intensity variation that exhibits multiple orientations. This makes the detection and characterization of junctions a challenging problem.

We analyze a variational approach to image segmentation that is based on a strictly convex non-quadratic cost functional. The smoothness term combines a standard first-order measure for image regions with a total-variation based measure for signal transitions. Accordingly, the costs associated with "discontinuities" are given by the length of level lines and local image contrast. For real images, this provides a reasonable approximation of the variational model of Mumford and Shah that has been suggested as a generic approach to image segmentation. The global properties of the convex variational model are favorable to applications: Uniqueness of the solution, continuous dependence of the solution on both data and parameters, consistent and efficient numerical approximation of the solution with the FEM-method. Various global and local properties of the convex variational model are analyzed and illustrated with numerical examples. Apart from the favorable global properties, the approach ...

This paper describes a knowledge-based approach to automatic 3D reconstruction of buildings from aerial imagery. The approach relies upon the use of 2D GIS map and knowledge about the imaging geometry and acquisition parameters. We show that by integrating knowledge from the images and the GIS map, the complexity of the building reconstruction process can be greatly reduced. In the first stage the buildings are localized in the images based on information about the ground planes of the buildings contained in the GIS map. This restricts the processing at all following stages to one building structure. Further, a building can be partitioned into more simple building parts, corresponding to some basic building models. By merging the separately reconstructed building parts the initial building model is found. The building reconstruction process is described as a tree search in the space of possible building hypotheses. 1

In this contribution we describe a method to assess the activity of vehicles based on airborne image sequences taken by an infrared camera. Active vehicles often appear as a configuration of a dark and a bright spot close to each other. The sensor movement is inferred from image sequences. Due to the fast velocity of the platform estimations of vehicle movements require a precise measurement of the sensor movement. The camera may be tilted with the aircraft giving arbitrarily oblique views. Camera movements are treated as projective 2D-homographies. For the search of a subset of image point correspondences that is free of outliers and gives a precise estimate of the movement we use a production system implementing good sample consensus (GSAC). This new method is derived from the well known RANSAC-decisions and improves them by preferring good samples to random samples. As assessment criterion for minimal samples the area of the smallest triangle in the sample is used. We motivate the criterion for the quality of samples by error propagation through the estimated homography. A comparison is made with other robust estimation techniques namly RANSAC and iterative re-weighted least squares. 1.1 Vehicle detection 1.

Airborne videos are gaining increasing importance. Video cameras are taking huge amounts of measurements for low costs. Their low weight and low requirement for energy makes them particularly attractive for small airborne carriers with low payload. Such carriers are discussed for military as well as for civil applications, e.g. traffic-surveillance. Often video cameras are used for documentation and reference in connection with other sensor systems. In addition to panchromatic or ordinary colour videos, nowadays also cameras operating in the thermal spectral domain gain attention. For the utilization of any stream of measurements taken from a moving platform the pose of the sensor in orientation and position has to be constantly determined. For airborne platforms often GPS and INS are used to acquire this information. However, the video stream itself provides also possibilities to estimate pose parameters. In this contribution we restrict our investigation to almost flat scenes but we allow oblique views both forward looking and side looking. The optical flow of the scene fixed structure on the world plane is estimated by a planar projective homography. This requires at least four point or line correspondences that can be traced over an appropriate number of frames. If the focal length is not changed and the camera has not been rotated, the proper transform will be restricted to a central collineation with five degrees of freedom. Two of these- giving the vertex or epipole- can be inferred directly from image correspondences. The remaining three are then estimated from the homography by solving a homogenous linear system. They give the axis or horizon, from which we obtain the rotational part of the pose, and a scale parameter for the speed to height ratio. Common level keeping flight manoeuvres where the epipole is close to the horizon lead to elations. Other manoeuvres- like e.g. landing- lead to

constraints This paper presents a knowledge-based approach for automatic 3D building reconstruction. By combining the aerial image analysis with information from GIS maps and domain specific knowledge the complexity of the building reconstruction process can be greatly reduced. The building reconstruction process is described as a tree search in the space of possible building hypotheses. To guide the search of the tree an evaluation function based on mutual information is defined. This evaluation function defines the score of matching between a hypothesized building model and the images. Thus, the evaluation function allows comparison of different building hypotheses, which are refined by applying a fitting algorithm. The performance of the 3D reconstruction is improved by incorporating geometric constraints. 1.

We present a knowledge-based system for automatic 3D building reconstruction from aerial images. Our approach relies on combining pairs of stereo images with 2D GIS map and domain knowledge. Since most buildings can be described as aggregation of simple building types, the domain knowledge is represented in a building library containing building primitives (flat, gable, and hip roof building). The approach of modeling buildings using a set of basic building models suggests the usage of Constructive Solid Geometry representation for building description. The building reconstruction process is formulated as a hypothesis generation and verification scheme. It starts with the partitioning of a building in simple building parts based on the ground plan defined in the map. For each building partition different building hypotheses are generated corresponding to the building primitives defined in the building library. The evaluation of the generated building models is based on the formulation of the mutual information between the model and the images. The CSG tree representing a building is given by the best fit of the building models corresponding to the building partitions. We used this method to reconstruct buildings in suburban and urban scenes. The method worked well even in difficult conditions (noise, shadow).

The advent of massive quantities of high spatial resolution multispectral imagery mandates the development of effective algorithms to manipulate these data. As a result of this research, an algorithm for automatic recognition of road signs in color terrestrial imagery has been developed. The main premise is to utilize prior knowledge about the color and shape of these signs to facilitate both hypothesis generation and verification stages of the recognition process. Any color can be obtained by combining the three basic colors (Red, Green and Blue) with different ratios. Knowing the colors of the targets of interest, one can define the range of response in the RGB bands of color imagery to yield this particular color. This will be the starting point for hypothesis generation of regions with the predefined colors. An interest operator is applied to extract corner points from the isolated regions. These corner points will be matched with the corner points associated with the different sig...

This paper describes recent work in the area of algorithm evaluation for a certain vision task - that of calibration - for a certain class of applications - that is of space applications. The work addresses the problem of the characterisation and the identification of the limitations on the performance of this vision task. This is necessary in order to permit the design and application of reliable calibration techniques, a characteristic which is essential in the space environment where the cost of failure is great, and the opportunity for human intervention is severely restricted. There has in general been very little published in the area of the performances of vision algorithms. This unfortunately makes it very hard to construct vision systems using the standard engineering basis of building complex systems from pre-proved and trusted modules. The aim of our work has been to address this problem directly. We have therefore constructed a test environment which brings together various...