A method is proposed to track the full hand motion from 3D points on the surface of the hand that were reconstructed and tracked using a stereoscopic set of cameras. This approach combines the advantages of previous methods that use 2D motion (e.g. optical flow), and those that use a 3D reconstruction at each time frame to capture the hand motion. Matching either contours or a 3D reconstruction against a 3D hand model is usually very di#cult due to self-occlusions and the locally-cylindrical structure of each phalanx in the model, but our use of 3D point trajectories constrains the motion and overcomes these problems.

Our purpose in this article is to superimpose a 3D model of the liver, its vessels and tumors (reconstructed from CT images) on external video images of the patient for hepatic surgery guidance. The main constraints are the robustness, the accuracy and the computation time. Because of the absence of visible anatomical landmarks and of the "cylindrical" shape of the upper abdomen, we used some radio-opaque fiducials. The classical least-squares method assuming that there is no noise on the 3D point positions, we designed a new Maximum Likelihood approach to account for this existing noise and we show that it generalizes the classical approaches. Experiments on synthetic data provide evidences that our new criterion is up to 20% more accurate and much more robust, while keeping a computation time compatible with realtime at 20 to 40 Hz. Eventually, careful validation experiments on real data show that an accuracy of 2 mm can be achieved within the liver.

Our goal is an automated 2D-image-pair registration algorithm capable of aligning images taken of a wide variety of natural and man-made scenes as well as many medical images. The algorithm should handle low overlap, substantial orientation and scale differences, large illumination variations, and physical changes in the scene. An important component of this is the ability to automatically reject pairs that have no overlap or have too many differences to be aligned well. We propose a complete algorithm including techniques for initialization, for estimating transformation parameters, and for automatically deciding if an estimate is correct. Keypoints extracted and matched between images are used to generate initial similarity transform estimates, each accurate over a small region. These initial estimates are rank-ordered and tested individually in succession. Each estimate is refined using the Dual-Bootstrap ICP algorithm, driven by matching of multiscale features. A three-part decision criteria, combining measurements of alignment accuracy, stability in the estimate, and consistency in the constraints, determines whether the refined transformation estimate is accepted as correct. Experimental results on a data set of 22 challenging image pairs show that the algorithm effectively aligns 19 of the 22 pairs and rejects 99.8 percent of the misalignments that occur when all possible pairs are tried. The algorithm substantially out-performs algorithms based on keypoint matching alone.

Abstract. Correlation is a very effective way to align intensity images. We extend the correlation technique to point set registration using a method we call kernel correlation. Kernel correlation is an affinity measure, and it is also a function of the point set entropy. We define the point set registration problem as finding the maximum kernel correlation configuration of the the two point sets to be registered. The new registration method has intuitive interpretations, simple to implement algorithm and easy to prove convergence property. Our method shows favorable performance when compared with the iterative closest point (ICP) and EM-ICP methods. 1

Abstract. In this paper we address the problem of aligning 3-D data with articulated shapes. This problem resides at the core of many motion tracking methods with applications in human motion capture, action recognition, medical-image analysis, etc. We describe an articulated and bending surface representation well suited for this task as well as a method which aligns (or registers) such a surface to 3-D data. Articulated objects, e.g., humans and animals, are covered with clothes and skin which may be seen as textured surfaces. These surfaces are both articulated and deformable and one realistic way to model them is to assume that they bend in the neighborhood of the shape’s joints. We will introduce a surface-bending model as a function of the articulated-motion parameters. This combined articulated-motion and surface-bending model better predicts the observed phenomena in the data and therefore is well suited for surface registration. Given a set of sparse 3-D data (gathered with a stereo camera pair) and a textured, articulated, and bending surface, we describe a register-and-fit method that proceeds as follows. First, the data-to-surface registration problem is formalized as a classifier and is carried out using an EM algorithm. Second, the data-to-surface fitting problem is carried out by minimizing the distance from the registered data points to the surface over the joint variables. In order to illustrate the method we applied it to the problem of hand tracking. A hand model with 27 degrees of freedom is successfully registered and fitted to a sequence of 3-D data points gathered with a stereo camera pair. 1

INTRODUCTION E PIDAURE is the name of a research project launched in 1989 at INRIA Rocquencourt, close to Paris, France. At that time, after a first experience of research in Computer Vision [1] in the group of O. Faugeras, I was very enthusiastic about the idea of transposing research results of digital image analysis into the medical domain. Visiting hospitals and medical research centers, I was progressively convinced that Medical Image Analysis was an important research domain by itself. In fact I had the impression that a better exploitation of the available medical imaging modalities would require more and more advanced image processing tools in the short and long-term future, not only to assess the diagnosis on more objective and quantitative measurements, but also to better prepare, control and evaluate the therapy. Fig. 1. This image has been the "Logo" of the Epidaure project for a long time. It was also used as a logo of the first CVRMed Conference held in Nice in 1

Abstract. We propose the use of a particle filter as a solution to the rigid shapebased registration problem commonly found in computer-assisted surgery. This approach is especially useful where there are only a few registration points corresponding to only a fraction of the surface model. Tests performed on patient models, with registration points collected during surgery, suggest that particle filters perform well and also provide novel quality measures to the surgeon. 1

Abstract — Medical Image Analysis plays a crucial role in the diagnosis, planning, control and follow-up of therapy. To be combined efficiently with medical robotics, Medical Image Analysis can be supported by the development of specific computational models of the human body operating at various levels. We describe in this article a hierarchy of these computational models, including the geometrical, physical and physiological levels, and illustrate their potential use in a number of advanced medical applications including image guided, robot-assisted and simulated medical interventions. We conclude this article with scientific perspectives.

In this paper we propose a general framework to solve the articulated shape matching problem, formulated as finding point-to-point correspondences between two shapes represented by 2-D or 3-D point clouds. The original pointsets are embedded in a spectral representation and the actual matching is carried out in the embedded space. We analyze the advantages of this choice as well as the reasons for which the task remains a difficult one. In particular, we show that although embedded-space matching still has intrinsic combinatorial difficulties, it can be solved by searching for an optimal orthogonal transformation that aligns the two shape embeddings. Relying on the model based clustering formalism, we propose a probabilistic formulation which casts the matching into an EM algorithm. Outliers are properly handled by the algorithm and a simple strategy is adopted to initialize it. Experiments are performed with three embedding methods (Isomap, LLE, and Laplacian embedding) and with 3-D voxelsets representing a human-motion sequence. 1.

Range sensors, such as laser range finder and stereo vision systems, return point-samples of a scene. Typical point-sampled vision problems include registration, regularization and merging. We introduce a robust distance minimization approach to solving the three classes of problems.