Model-based recognition and motion tracking depends upon the ability to solve for projection and model parameters that will best fit a 3-D model to matching 2-D image features. This paper extends current methods of parameter solving to handle objects with arbitrary curved surfaces and with any number of internal parameters representing articulations, variable dimensions, or surface deformations. Numerical

Model-based object recognition systems have rarely dealt directly with 3D perspective while matching models to images. The algorithms presented here use 3D pose recovery during matching to explicitly and quantitatively account for changes in model appearance associated with 3D perspective. These algorithms use random-start local search to find, with high probability, the globally optimal correspondence between model and image features in spaces containing over 2 100 possible matches. Three specific algorithms are compared on robot landmark recognition problems. A fullperspective algorithm uses the 3D pose algorithm in all stages of search while two hybrid algorithms use a computationally less demanding weak-perspective procedure to rank alternative matches and updates 3D pose only when moving to a new match. These hybrids successfully solve problems involving perspective, and in less time than required by the full-perspective algorithm.

Our purpose in this article is to superimpose a 3D model of the liver, its vessels and tumors (reconstructed from CT images) on external video images of the patient for hepatic surgery guidance. The main constraints are the robustness, the accuracy and the computation time. Because of the absence of visible anatomical landmarks and of the "cylindrical" shape of the upper abdomen, we used some radio-opaque fiducials. The classical least-squares method assuming that there is no noise on the 3D point positions, we designed a new Maximum Likelihood approach to account for this existing noise and we show that it generalizes the classical approaches. Experiments on synthetic data provide evidences that our new criterion is up to 20% more accurate and much more robust, while keeping a computation time compatible with realtime at 20 to 40 Hz. Eventually, careful validation experiments on real data show that an accuracy of 2 mm can be achieved within the liver.

This paper presents a new technique for reconstructing the 3D structure and motion of a scene undergoing relative rotational motion with respect to the camera. Given image correspondences of point features tracked over many frames, a two--stage technique for reconstruction is presented. First, a grouping algorithm is developed which exploits spatio--temporal constraints of the common motion to achieve a reliable description of discrete point correspondences as curved trajectories (general conics in the case of rotational motion) in the image plane. In contrast, trajectories fitted to points independent of each other lead to arbitrary image descriptions and very inaccurate 3D parameters. Second, a new closed--form solution, under perspective projection, for the 3D motion and location of points from the computed image trajectories is presented. Both stages are applied to real image sequences with good results. This approach represents a first step in a longer--term research eff...

A least-squares method simultaneously solves for the model-to-sensor-suite pose and sensor-to-sensor registration. The development is for a sensor-suite containing separate range and optical sensors. To address outliers and, more generally, match finding, a statistical method (median filtering) and a search method (local search) are developed. Sensitivity to Gaussian noise and the choice of initial pose estimates is investigated on synthetic data. Both of the matching methods are demonstrated on real data. 1 Introduction Coregistration describes a process which simultaneously refines both the estimated 3D pose of an object relative to a sensor suite as well as the registration parameters relating the coordinate systems of a range sensor and an optical sensor. It extends single sensor pose work [14, 17, 9, 19] by imposing contraints on both sensor and object geometry. Coregistration will support multi-sensor object verification but is not itself an object recognition tool. It presuppo...

1 This report summarizes progress in image understanding research at the University of Massachusetts over the past year. Many of the individual efforts discussed in this paper are further developed in other papers in this proceedings. The summary is organized into several areas: 1. Mobile Robot Navigation 2. Motion and Stereo Processing 3. Knowledge-Based Interpretation of Static Scenes 4. Image Understanding Architecture The research program in computer vision at UMass has as one of its goals the integration of a diverse set of research efforts into a system that is ultimately intended to achieve real-time image interpretation in a variety of vision applications. 1. Mobile Robot Navigation The initial focus of the mobile robot navigation project (Fennema and Hanson 1990b) has been on the development of a system for goal oriented navigation through a partially modeled, unchanging environment which contains no unmodeled obstacles. This simplified environment is intended to provide a fou...

Most Automatic Target Recognition (ATR) algorithms operate in 2D image space. Even when using 3D models, these 3D models are typically translated off-line into sets of 2D representations, such as templates, which are then applied to imagery to perform detection, recognition and verification. In contrast to this approach, the work reported here takes steps toward direct matching of 3D models to range and optical imagery. The key idea is to exploit known 3D sensor and target geometry to drive a model-based sensor fusion process. This process, which we call `coregistration', resolves uncertainty in the 3D placement of the target relative to the sensors as well as uncertainty in the exact pixel registration between range and optical sensors. A specific coregistration algorithm is presented along with results on both synthetic and real data. Extending coregistration, two approaches are presented for finding locally optimal matches between 3D target models and multi-sensor data. Both are dem...

Determination of relative three-dimensional (3--D) position, orientation, and relative motion between two reference frames is an important problem in robotic guidance, manipulation, and assembly as well as in other fields such as photogrammetry. A solution to this problem that uses two-dimensional (2--D), intensity images from a single camera is desirable for real-time applications. Where the object geometry is unknown, the estimation of structure is also required. A single camera is advantageous because a standard video camera is low in cost, setup and calibration are simple, physical space requirements are small, reliability is high, and low-cost hardware is available for digitizing and processing the images. A di#culty in performing this measurement is the process of projecting 3--D object features to 2--D images, a nonlinear transformation. Noise is present in the form of perturbations to the assumed process dynamics, imperfections in system modeling, and errors in the feature locations extracted from the 2--D images. This dissertation presents solutions to the remote measurement problem for a dynamic system given a sequence of 2--D intensity images of an object where feature positions of the object are known relative to a base reference frame and where the feature positions are unknown relative to a base reference frame. The 3--D transformation is modeled as a nonlinear stochastic system with the state estimate providing six degree-of-freedom motion and position values. The stochastic model uses the iterated extended Kalman filter as an estimator and as a screw representation of the 3--D transformation based on dual quaternions. Dual quaternions provide a means to represent both rotation and translation in a unified notation. The method has been implemented and tes...

The complementary nature of LADAR, FLIR and color data for ATR is being exploited by new algorithms in a three stage recognition system. The stages are initial detection, target class and pose hypothesis generation, and precise model to multisensor coregistration matching. Coregistration globally aligns 3D target models with range, IR and color imagery while simultaneously refining registration parameters between sensors. This model directed approach is expected to improve ATR performance for occluded targets, targets seen at unusual angles, and targets in cluttered settings. Color is used for initial target detection under daylight conditions and camouflage learned from training generalizes across vehicles and distinguishes targets from natural terrain. Target class and pose hypothesis generation will draw upon existing LADAR boundary matching work extended to tolerate more occlusion, clutter and viewpoint variation. New model to multisensor coregistration algorithms appear robust in...

Our purpose is to provide an augmented reality system for RadioFrequency (RF) tumor ablation guidance that could superimpose a 3D model of the liver, its vessels and tumors (reconstructed from CT images) on external video images of the patient. The main constraints are the reliability and the accuracy, which justies a 3D/2D registration based on radio-opaque ducials rather than surfaceor 3D- based techniques. Then, a lack in the statistical assumptions of the classical 3D/2D registration methods led us to the derivation of a new extended 3D/2D criterion. Careful validation experiments on real data show that an accuracy of 2 mm can be achieved in clinically relevant conditions, and that our new criterion is up to 9% more accurate, while keeping a computation time compatible with real-time at 20 to 40 Hz. However, the accuracy of the registration strongly vary w.r.t. the experimental conditions (cameras angle, number of markers...). Thus, to provide a safe system, we should supply an error prediction that take this parameters into account. Propagating the data noise through both our criterion and the classical one, we obtain an explicit formulation of the registration error. As the real conditions do not always t the theory, it is critical to validate our prediction with real data. Thus, we perform a rigorous incremental validation of each assumption using successively: synthetic data, real video images of a precisely known object, and nally real CT and video images of a soft phantom. Results point out that our error prediction is fully valid in our application range. Eventually, we provide an accurate RA guidance system that allows the automatic detection of potentially inaccurate guidance.