Abstract The objective of this thesis is to develop algorithms and software tools for acquiring a structured scene representation from real imagery which can be used for 2-D video coding as well as for interactive 3-D presentation. A new approach will be introduced that recovers visible 3-D structure and texture from one or more partially overlapping uncalibrated views of a scene composed of straight edges. The approach will exploit CAD-like geometric properties which are invariant under perspective projection and are readily identifiable among detectable features in images of man-made scenes (i.e. scenes that contain right angles, parallel edges and planar surfaces). In certain cases, such knowledge of 3-D geometric relationships among detected 2-D features allows recovery of scene structure, intrinsic camera parameters, and camera rotation/position from even a single view. Model-based representations of video offer the promise of ultra-low bandwidths by exploiting the redundancy of information contained in sequences which visualize what is largely a static 3-D world. An algorithm will be proposed that uses a previously recovered estimate of a scene's 3-D structure and texture for the purpose of model-based video coding. Camera focal length and pose will be determined for each frame in a video sequence taken in that known scene. Unmodeled elements (e.g. actors, lighting effects, and noise) will be detected and described in separate compensating signals.

The richness of natural images makes the quest for optimal representations in image processing and computer vision challenging. The latter observation has not prevented the design of image representations, which trade off between efficiency and complexity, while achieving accurate rendering of smooth regions as well as reproducing faithful contours and textures. The most recent ones, proposed in the past decade, share an hybrid heritage highlighting the multiscale and oriented nature of edges and patterns in images. This paper presents a panorama of the aforementioned literature on decompositions in multiscale, multi-orientation bases or dictionaries. They typically exhibit redundancy to improve sparsity in the transformed domain and sometimes its invariance with respect to simple geometric deformations (translation, rotation). Oriented multiscale dictionaries extend traditional wavelet processing and may offer rotation invariance. Highly redundant dictionaries require specific algorithms to simplify the search for an efficient (sparse) representation. We also discuss the extension of multiscale geometric decompositions to non-Euclidean domains such as the sphere or arbitrary meshed surfaces. The etymology of panorama suggests an overview, based on a choice of partially overlapping “pictures”.

We present an algorithm for efficient stratified importance sampling of environment maps that generates samples in the positive hemisphere defined by local orientation of arbitrary surfaces while accounting for cosine weighting. The importance function is dynamically adjusted according to the surface normal using steerable basis functions. The algorithm is easy to implement and requires no user-defined parameters. As a preprocessing step, we approximate the incident illumination from an environment map as a continuous piecewise linear function on S 2 and represent this as a triangulated height field. The product of this approximation and a dynamically orientable steering function, viz. the cosine lobe, serves as an importance sampling function. Our method allows the importance function to be sampled with an asymptotic cost of O(logn) per sample where n is the number of triangles. The most novel aspect of the algorithm is its ability to dynamically compute normalization factors which are integrals of the illumination over the positive hemispheres defined by the local surface normals during shading. The key to this feature is that the weight variation of each triangle due to the clamped cosine steering function can be well approximated by a small number of spherical harmonic coefficients which can be accumulated over any collection of triangles, in any orientation, without introducing higher-order terms. Consequently, the weighted integral of the entire steerable piecewise-linear approximation is no more costly to compute than that of a single triangle, which makes re-weighting and re-normalizing with respect to any surface orientation a trivial constant-time operation. The choice of spherical harmonics as the set of basis functions for our steerable importance function allows for easy rotation between coordinate systems. Another novel element of our algorithm is an analytic parametrization for generating stratified samples with linearly-varying density over a triangular support. 1

We present an algorithm to segment texture images based on a novel middle-level vision approach to compute global image structures. Previous schemes to compute global structures are not truly global on the discrete image, either because they are discrete approximations to global continuous processes or because some other non-global component is needed to make the algorithms functional. Our scheme, Curved Inertia Frames, is based on a novel parallel architecture built around non-cartesian networks. We contend that cartesian networks are specially unsuitable for vision because they have biased orientations, can not provide local estimates of curvature, and can not concentrate processing on selected image locations. Our non-cartesian network proposals solves this three problems and enables a truly global computation. Our scheme, Curved Inertia Frames, has two stages. In the first stage, we compute local estimates for the texture gradient by filtering the image with multiple filters (simil...

We describe a new approach for focusing images obtained by timefrequency analysis of speech signals. Our summary consists of a set of curves which include the formants. Due to the oscillatory nature of the two dimensional data, all the classical approaches to this problem have shortcomings. Our method is semi-automatic, requiring as input only the maximum number of curves. The summary curves are designed to correspond with human perception: they are not allowed to cross, they can die with time, new curves can be born, or the same curve can die and then be reborn. Our approach is based on two modeling assumptions a) we view each normalized image as a probability density function, and b) we approximate these densities, conditional on time, as mixtures of Gaussians. The curves then represent the means of the Gaussians as a function of time. Within this framework, we fit the model by maximum cross-entropy, and enforce the constraints by adapting the standard fitting algorithm. Keywords: c...

. This work presents a SAR image segmentation scheme consisting of a sequence of four modules, all selected from the literature. These modules are: i) a speckle model-free contour detector that is the core of the segmentation scheme; ii) a geometrical procedure to detect closed regions from non-connected contours; iii) a region growing procedure whose merging rules exploit local image properties, both topological and spectral, to eliminate artifacts and reduce oversegmentation introduced by the second stage; iv) a neural network clustering algorithm to detect global image regularities in the sequence of within-segment properties extracted from the partitioned image provided by the third stage. In the framework of a commercial image-processing software toolbox, the proposed SAR image segmentation scheme employs a contour detector that is promising because: i) it is easy to use, requiring the user to select only one contrast threshold as a relative number; and ii) it exploits no prior do...

We develop e#cient techniques for the non-rigid registration of medical images by using representations that adapt to the anatomy found in such images.

This paper describes a hierarchical algorithm for multi-resolution texture-based image enhancement for segmentation performance. The proposed algorithm is a modification of the steerable pyramid algorithm that has been successftully implemented for research in texture synthesis. ' We investigate in this paper the application of such a multiscale approach for image analysis to the problem of image enhancement for segmentation. We present the application of the proposed algorithm to two medical images as a first exploration of the potential benefits of the approach. While some quantitative measure of assessment needs ultimately to be developed for performance assessment, some subjective evaluation at this early stage of the research can provide early feedback on the magnitude of a potential benefit. While the early results presented in this paper are encouraging, it is yet to be demonstrated that the proposed algorithm or an optimized version of the algorithm yields increased performance compared to other techniques.