Accurate registration between real and virtual objects is crucial for augmented reality applications. Existing tracking methods are individually inadequate: magnetic trackers are inaccurate, mechanical trackers are cumbersome, and vision-based trackers are computationally problematic. We present a hybrid tracking method that combines the accuracy of vision-based tracking with the robustness of magnetic tracking without compromising real-time performance or usability. We demonstrate excellent registration in three sample applications.

Abstract—Camera calibration and the acquisition of Euclidean 3D measurements have so far been considered necessary requirements for overlaying three-dimensional graphical objects with live video. In this article, we describe a new approach to videobased augmented reality that avoids both requirements: It does not use any metric information about the calibration parameters of the camera or the 3D locations and dimensions of the environment’s objects. The only requirement is the ability to track across frames at least four fiducial points that are specified by the user during system initialization and whose world coordinates are unknown. Our approach is based on the following observation: Given a set of four or more noncoplanar 3D points, the projection of all points in the set can be computed as a linear combination of the projections of just four of the points. We exploit this observation by 1) tracking regions and color fiducial points at frame rate, and 2) representing virtual objects in a non-Euclidean, affine frame of reference that allows their projection to be computed as a linear combination of the projection of the fiducial points. Experimental results on two augmented reality systems, one monitor-based and one head-mounted, demonstrate that the approach is readily implementable, imposes minimal computational and hardware requirements, and generates real-time and accurate video overlays even when the camera parameters vary dynamically. Index Terms—Augmented reality, real-time computer vision, calibration, registration, affine representations, feature tracking, 3D interaction techniques. 1

Augmented reality deals with the problem of dynamically augmenting or enhancing (images or live video of) the real world with computer generated data (e.g., graphics of virtual objects). This poses two major problems: (a) determining the precise alignment of real and virtual coordinate frames for overlay, and (b) capturing the 3D environment including camera and object motions. The latter is important for interactive augmented reality applications where users can interact with both real and virtual objects. Here we address the problem of accurately tracking the 3D motion of a monocular camera in a known 3D environment and dynamically estimating the 3D camera location. We utilize fully automated landmark-based camera calibration to initialize the motion estimation and employ extended Kalman filter techniques to track landmarks and to estimate the camera location. The implementation of our approach has been proven to be efficient and robust and our system successfully tracks in real-tim...

We present a color-video-based augmented reality (AR) system that is designed to be self-tracking, that is, it requires no separate tracking subsystem. Rather, tracking is performed strictly from the video images acquired through the lens of the camera also used to view the real world. The methods for tracking are rooted in prior research in photogrammetry and computer vision. This approach to tracking for AR systems enables a variety of new applications in assembly guidance that are not feasible with current AR technology. Our initial application is in aircraft manufacturing. We outline our approaches to feature detection, correspondence, pose determination, and system calibration. The results obtained thus far are summarized along with the problems we encountered.

Augmented reality is a term used to describe systems in which computer-generated information is superimposed on top of the real world; for example, through the use of a see-through head-mounted display. A human user of such a system could still see and interact with the real world, but have valuable additional information, such as descriptions of important features or instructions for performing physical tasks, superimposed on the world. For example, the computer could identify objects and overlay them with graphic outlines, labels, and schematics. The graphics are registered to the real-world objects and appear to be painted onto those objects. Augmented reality systems can be used to make productivity aids for tasks such as inspection, manufacturing, and navigation. One of the most critical requirements for augmented reality is to recognize and locate real-world objects with respect to the persons head. Accurate registration is necessary in order to overlay graphics accurately on top of the real-world objects. At the Colorado School of Mines, we have developed a prototype augmented reality system that uses head-mounted cameras and computer vision techniques to accurately register the head to the scene. The current system locates and tracks a set of preplaced passive fiducial targets placed on the real-world objects. The system computes the pose of the objects and displays graphics overlays using a see-through head-mounted display. This paper describes the architecture of the system and outlines the computer vision techniques used. Keywords: augmented reality, registration, computer vision, pose estimation, fiducials, head-mounted displays 1.

We present several techniques for producing two visual and modeling e#ects in augmentedreality. The #rst e#ect involves interactively calculating the occlusions between real and virtual objects. The second e#ect utilizes acollision detection algorithm to automatically move dynamic virtual objects until they come in contact with static real objects in augmentedreality. All of the techniques utilize calibrated data derivedfrom images of a real-world environment. 1. Introduction Augmented reality #AR# is a combination of technologies distinct from virtual reality #VR#, that promises to support a wider range of applications. Interest in AR has substantially increased in the past few years, with research groups exploring diagnostic, manufacturing, medical and repair applications 1 . In augmented reality, the computer provides additional visual information that enhances or augments a user's view of the real world. Instead of replacing the world with a completely virtual environment, as i...

AbstractÐA method is developed to analyze the accuracy of the relative head-to-object position and orientation (pose) in augmented reality systems with head-mounted displays. From probabilistic estimates of the errors in optical tracking sensors, the uncertainty in head-to-object pose can be computed in the form of a covariance matrix. The positional uncertainty can be visualized as a 3D ellipsoid. One useful benefit of having an explicit representation of uncertainty is that we can fuse sensor data from a combination of fixed and head-mounted sensors in order to improve the overall registration accuracy. The method was applied to the analysis of an experimental augmented reality system, incorporating an optical see-through head-mounted display, a head-mounted CCD camera, and a fixed optical tracking sensor. The uncertainty of the pose of a movable object with respect to the head-mounted display was analyzed. By using both fixed and head mounted sensors, we produced a pose estimate that is significantly more accurate than that produced by either sensor acting alone. Index TermsÐAugmented reality, pose estimation, registration, uncertainty analysis, error propagation, calibration.

In an augmented reality system, it is required to obtain the position and orientation of the user's viewpoint to display the composed image maintaining correct registration of real and virtual worlds. All the procedures must be done in real-time. This paper proposes a method for augmented reality with a stereo vision sensor and video see-through head mounted display. It can synchronize the display-timing between the virtual and real worlds, so that an alignment error is reduced. The method calculates camera parameters from three markers in image sequences captured by a pair of stereo cameras mounted on a HMD. In addition, it estimates depth of real world from a pair of stereo images to generate a composed image maintaining consistent occlusions between real and virtual objects. A region of depth estimation is efficiently limited by calculating a position of virtual object using camera parameters. Finally, we have developed a video see-through augmented reality system which mainly consists of a pair of stereo cameras mounted on the HMD and a standard graphics workstation. The feasibility of the system has been successfully demonstrated with experiments.

Augmented reality (AR) constitutes a very promising new user interface concept for many applications. In this chapter, we pay particular attention to developing AR technology for exterior construction applications, augmenting video sequences from construction sites with information stored in models. Such augmentations can tremendously benefit several business processes common to many construction projects.

. Augmented reality #AR# is a technology in which a user's view of the real world is enhanced or augmented with additional information generated from a computer model. Using AR technology, users can interact with a combination of real and virtual objects in a natural way. This paradigm constitutes the core of a very promising new technology for many applications. However, before it can be applied successfully, AR has to ful#ll very strong requirements including precise calibration, registration and tracking of sensors and objects in the scene, as well as a detailed overall understanding of the scene. At ECRCwe see computer vision and image processing technology play an increasing role in acquiring appropriate sensor and scene models. To balance robustness with automation, weintegrate automatic image analysis with both interactive user assistance and input from magnetic trackers and CAD-models. Also, in order to meet the requirements of the emerging global information society...