We describe techniques for coordinating the actions of large numbers of small-scale robots to achieve useful large-scale results in surveillance, reconnaissance, hazard detection, and path finding. We exploit the biologically inspired notion of a "virtual pheromone," implemented using simple transceivers mounted atop each robot. Unlike the chemical markers used by insect colonies for communication and coordination, our virtual pheromones are symbolic messages tied to the robots themselves rather than to fixed locations in the environment. This enables our robot collective to become a distributed computing mesh embedded within the environment, while simultaneously acting as a physical embodiment of the user interface. This leads to notions of world-embedded computation and world-embedded displays that provide different ways to think about robot colonies and the types of distributed computations that such colonies might perform.

The biggest single obstacle to building effective augmented reality (AR) systems is the lack of accurate wide-area sensors for trackers that report the locations and orientations of objects in an environment. Active (sensor-emitter) tracking technologies require powereddevice installation, limiting their use to prepared areas that are relatively free of natural or man-made interference sources. Vision-based systems can use passive landmarks, but they are more computationally demanding and often exhibit erroneous behavior due to occlusion or numerical instability. Inertial sensors are completely passive, requiring no external devices or targets, however, the drift rates in portable strapdown configurations are too great for practical use. In this paper, we present a hybrid approach to AR tracking that integrates inertial and vision-based technologies. We exploit the complementary nature of the two technologies to compensate for the weaknesses in each component. Analysis and experimental...

A novel framework enables accurate AR registration with integrated inertial gyroscope and vision tracking technologies. The framework includes a two-channel complementary motion filter that combines the lowfrequency stability of vision sensors with the highfrequency tracking of gyroscope sensors, hence, achieving stable static and dynamic six-degree-of-freedom pose tracking. Our implementation uses an Extended Kalman filter (EKF). Quantitative analysis and experimental results show that the fusion method achieves dramatic improvements in tracking stability and robustness over either sensor alone. We also demonstrate a new fiducial design and detection system in our example AR annotation systems that illustrate the behavior and benefits of the new tracking method.

The biggest single obstacle to building effective augmented reality (AR) systems is the lack of accurate wide-area sensors for tracking the locations and orientations of objects in an environment. Active (sensor-emitter) tracking technologies require powered-device installation, limiting their use to prepared areas that are relatively free of natural or man-made interference sources. Vision-based systems can use passive landmarks, but they are more computationally demanding and often exhibit erroneous behavior due to occlusion or numerical instability. Inertial sensors are completely passive, requiring no external devices or targets, however, their drift rates in portable strapdown configurations are too great for practical use. In this paper, we present a hybrid approach to orientation tracking that integrates inertial and vision-based sensing. We exploit the complementary nature of the two technologies to compensate for the weaknesses in each component. Analysis and experimental results demonstrate the effectiveness of this approach.

Figure 1: (Left) A user operating a handheld augmented reality unit tracked in an urban environment. (Middle) Live shot showing the unit tracking a building. (Right) Screenshot from a pose close to the left images with overlaid building outline. This paper presents a model-based hybrid tracking system for outdoor augmented reality in urban environments enabling accurate, realtime overlays for a handheld device. The system combines several well-known approaches to provide a robust experience that surpasses each of the individual components alone: an edge-based tracker for accurate localisation, gyroscope measurements to deal with fast motions, measurements of gravity and magnetic field to avoid drift, and a back store of reference frames with online frame selection to re-initialise automatically after dynamic occlusions or failures. A novel edge-based tracker dispenses with the conventional edge model, and uses instead a coarse, but textured, 3D model. This yields several advantages: scale-based detail culling is automatic, appearance-based edge signatures can be used to improve matching and the models needed are more commonly available. The accuracy and robustness of the resulting system is demonstrated with comparisons to map-based ground truth data.

Many Augmented Reality applications require accurate tracking. Existing tracking techniques require prepared environments to ensure accurate results. This paper motivates the need to pursue Augmented Reality tracking techniques that work in unprepared environments, where users are not allowed to modify the real environment, such as in outdoor applications. Accurate tracking in such situations is difficult, requiring hybrid approaches. This paper summarizes two 3DOF results: a real-time system with a compass -- inertial hybrid, and a non-real-time system fusing optical and inertial inputs. We then describe the preliminary results of 5- and 6-DOF tracking methods run in simulation. Future work and limitations are described.

The research project "NEXUS" aims at the development of a generic platform that supports location aware applications with mobile users. It is currently being carried out within a so-called research group supported by the DFG (German Research Council) in cooperation between the Institute for Photogrammetry, the Institute of Parallel and Distributed High-Performance Systems and the Institute of Communication Networks and Computer Engineering of the University of Stuttgart.

A new method for registration in augmented reality (AR) was developed that simultaneously tracks the position, orientation, and motion of the user's head, as well as estimating the three-dimensional (3-D) structure of the scene. The method fuses data from headmounted cameras and head-mounted inertial sensors. Two Extended Kalman Filters (EKF) are used; one of which estimates the motion of the user's head and the other that estimates the 3-D locations of points in the scene. A recursive loop is used between the two EKFs. The algorithm was tested using a combination of synthetic and real data, and in general was found to perform well. A further test showed that a system using two cameras performed much better than a system using a single camera, although improving the accuracy of the inertial sensors can partially compensate for the loss of one camera. The method is suitable for use in completely unstructured and unprepared environments. Unlike previous work in this area, this method requires no a priori knowledge about the scene, and can work in environments where the objects of interest are close to the user. Index terms: Augmented reality, pose estimation, registration, Kalman filter, structure from motion, computer vision, inertial sensors Submitted to Presence: Teleoperators and Virtual Environments 3 1

This paper presents user interface technology, using a glove based menuing system and 3D interaction techniques. ... It is designed to support applications that allow users to construct simple models of outdoor structures. The construction of models is performed using various 3D virtual reality interaction techniques, as well as using real time constructive solid geometry, to allow users to build up shapes with no prior knowledge of the environment. Previous work in virtual environments has tended to focus mostly on selection and manipulation, but not starting from an empty world. We demonstrate our user interface with the Tinmith-Metro application, designed to capture in city models and street furniture.

Human interaction with large numbers of robots or distributed sensors presents a number of difficult challenges including supervisory management, monitoring of individual and collective state, and apprehending situation awareness. A rich source of information about the environment can be provided even with robots that have no explicit representations or maps of their locale. To do this, we transform a robot swarm into a distributed interface embedded within the environment. Visually, each robot acts like a pixel within a much larger visual display space so that any robot need only communicate a small amount of information from its current location. Our approach uses Augmented Reality techniques for communicating information to humans from large numbers of small-scale robots to enable situation awareness, monitoring, and control for surveillance, reconnaissance, hazard detection, and path finding.