. The proliferation ofcomputing into the physical world promises more than the ubiquitous availability of computing infrastructure; it suggests new paradigms of interaction inspired by constant access to information and computational capabilities. For the past decade, applicationdriven research in ubicomp has pushed three interaction themes: natural interfaces, context-aware applications, and automated capture and access. To chart a course for future research in ubiquitous computing, we review the accomplishments of these efforts and point to remaining research challenges. Research in ubiquitous computing implicitly requires addressing some notion of scale; whether in the number and type of devices, the physical space of distributed computing or the number of people using a system. We posit a new area of applications research, everyday computing, focussed on scaling interaction with respect to time. Just as pushing the availability of computing away from the traditional desktop fun...

The role of computers in the modern office has tended to split our activities between virtual interactions in the realm of the computer and physical interactions with real objects that have been part of the traditional office infrastructure. This paper discusses a variety of scenarios we have implemented in which the physical world can be invisibly and seamlessly augmented with electronic tags in order to connect physical objects with virtual representations or computational functionality. We demonstrate the utility of linking physical objects to electronic services and actions that are naturally associated with their form (e.g., a dictionary and a web-based translation service). Unlike previous work in this area, we have focused on uniquely combining four inexpensive technologies with off-the-shelf applications, everyday objects, and computational devices. KEYWORDS: RFID tag, ubiquitous computing, tangible interface, physical UI, phicon, augmented reality. INTRODUCTION Six years ag...

For more than thirty years, people have relied primarily on screen-based text and graphics to interact with computers. Whether the screen is placed on a desk, held in one's hand, worn on one's head, or embedded in the physical environment, the screen has cultivated a predominantly visual paradigm of human-computer interaction. In this chapter, we discuss a growing space of interfaces in which physical objects play a central role as both physical representations and controls for digital information. We present an interaction model and key characteristics for such "tangible user interfaces," and explore these characteristics in a number of interface examples. This discussion supports a newly integrated view of both recent and previous work, and points the way towards new kinds of computationally-mediated interfaces that more seamlessly weave together the physical and digital worlds.

www.dgp.toronto.edu Recent advances in sensing technology have enabled a new generation of tabletop displays that can sense multiple points of input from several users simultaneously. However, apart from a few demonstration techniques [17], current user interfaces do not take advantage of this increased input bandwidth. We present a variety of multifinger and whole hand gestural interaction techniques for these displays that leverage and extend the types of actions that people perform when interacting on real physical tabletops. Apart from gestural input techniques, we also explore interaction and visualization techniques for supporting shared spaces, awareness, and privacy. These techniques are demonstrated within a prototype room furniture layout application, called RoomPlanner.

DiamondSpin is a toolkit for the efficient prototyping of and experimentation with multi-person, concurrent interfaces for interactive shared displays. In this paper, we identify the fundamental functionality that tabletop user interfaces should embody, then present the toolkit’s architecture and API. DiamondSpin provides a novel realtime polar to Cartesian transformation engine that has enabled new, around-the-table interaction metaphors to be implemented. DiamondSpin allows arbitrary document positioning and orientation on a tabletop surface. Polygonal tabletop layouts such as rectangular, octagonal, and circular tabletops can easily be constructed. DiamondSpin also supports multiple work areas within the same digital tabletop. Multi-user operations are offered through multithreaded input event streams, multiple active objects, and multiple concurrent menus. We also discuss insights on tabletop interaction issues we have observed from a set of applications built with DiamondSpin.

In this paper we present a system that electromagnetically tracks the positions and orientations of multiple wireless objects on a tabletop display surface. The system offers two types of improvements over existing tracking approaches such as computer vision. First, the system tracks objects quickly and accurately without susceptibility to occlusion or changes in lighting conditions. Second, the tracked objects have state that can be modified by attaching physical dials and modifiers. The system can detect these changes in realtime. We present several new interaction techniques developed in the context of this system. Finally, we present two applications of the system: chemistry and system dynamics simulation. Keywords Tangible user interface, interactive surface, object tracking, two-handed manipulation, system dynamics, augmented reality

To address Weiser’s human-centered vision of ubiquitous computing, the authors focus on physical interaction, general application features, and theories of design and evaluation for this new mode of human–computer interaction.

Augmented realityentails the use of models and their associated renderings to supplement information in a real scene. In order for this information to be relevant or meaningful, the models must be positioned and displayed in suchaway that they blend into the real world in terms of alignments, perspectives, illuminations, etc. For practical reasons the information necessary to obtain this realistic blending cannot be known a priori, and cannot be hard-wired into a system. Instead a number of calibration procedures are necessary so that the location and parameters of each of the system components are known. In this paper we identify the calibration steps necessary to build a computer model of the real world and then, using the monitor-based augmented reality system developed at ECRC#Grasp# as an example, we describe each of the calibration processes. These processes determine the internal parameters of our imaging devices #scan converter, frame grabber, and video camera#, as we...

In this paper, we introduce a fast and robust method for tracking positions of the centers and the fingertips of both right and left hands. Our method makes use of infraredcamera images for reliable detection of user's hands, and uses template matching strategy for finding fingertips. This method is an essential part of our augmented desk interface in which a user can, with natural hand gestures, simultaneously manipulate both physical objects and electronically projected objects on a desk, e.g., a textbook and related WWW pages. Previous tracking methods which are typically basedoncolor segmentation or background subtraction simply do not perform well in this type of application because an observedcolor of human skin and image backgrounds may change significantly due to projection of various objects onto a desk. In contrast, our proposed method was shown to be effective even in such a challenging situation through demonstration in our augmented desk interface. This paper describes the...

This thesis establishes a stochastic framework for tracking curves in visual clutter, using a Bayesian random-sampling algorithm. The approach is rooted in ideas from statistics, control theory and computer vision. The problem is to track outlines and features of foreground objects, modelled as curves, as they move in substantial clutter, and to do it at, or close to, video frame-rate. The algorithm, named Condensation, for Conditional density propagation, has recently been derived independently by several researchers, and is generating signi cant interest in the statistics and signal processing communities. This thesis contributes to the literature on Condensation-like lters by presenting some novel applications of and extensions to the basic algorithm, and contributes to the visual motion estimation literature by demonstrating high tracking performance in cluttered environments. Despite its power the Condensation algorithm has a remarkably simple form and this allows the use of non-linear motion models which combine characteristics of discrete Hidden Markov Models with the continuous Auto-Regressive Process motion models traditionally used in Kalman lters. These mixed discrete-continuous models have promising applications to the emerging eld of perception of action. This thesis also implements two algorithms to smooth the output of the Condensation lter which improves the accuracy of motion estimation in a batch-mode procedure after tracking is complete.