Multi-user, touch-sensing input devices create opportunities for the use of cooperative gestures – multi-user gestural interactions for single display groupware. Cooperative gestures are interactions where the system interprets the gestures of more than one user as contributing to a single, combined command. Cooperative gestures can be used to enhance users ’ sense of teamwork, increase awareness of important system events, facilitate reachability and access control on large, shared displays, or add a unique touch to an entertainment-oriented activity. This paper discusses motivating scenarios for the use of cooperative gesturing and describes some initial experiences with CollabDraw, a system for collaborative art and photo manipulation. We identify design issues relevant to cooperative gesturing interfaces, and present a preliminary design framework. We conclude by identifying directions for future research on cooperative gesturing interaction techniques. Author Keywords Cooperative gestures, gestures, computer-supported

this paper, we develop and articulate a set of design principles for constructing - in a systematic and extensible manner - multi-hand gestures on touch surfaces that can sense multiple points and shapes, and can also accommodate conventional point-based input

One way to build a remote sketching system is to use a video camera to image what each user draws at their site, transmit the video to the other sites, and display it there using an LCD projector. Such camera-projector based remote sketching systems date back to Paul Wellner's (largely unimplemented) Xerox Double DigitalDesk. To make such a system usable, however, the users have to be able to move the paper on which they are drawing, they have to be able to interact with the system using a convenient interface, and sketching sessions must be stored in a compact format so that they can be replayed later. We have recently developed Tele-Graffiti, a remote sketching system with the following three features: (1) real-time paper tracking to allow the users to move their paper during system operation, (2) a hand based user interface, and (3) automatic session summarization and playback. In this paper, we describe the design, implementation, and performance of Tele-Graffiti.

Arm and body pose are useful cues for deictic reference-- users naturally extend their arms to objects of interest in a dialog. We present some recent progress on untethered sensing of articulated arm and body configuration using robust stereo vision techniques. These techniques allow robust, accurate, real-time tracking of 3-D position and orientation. We demonstrate users' performance with our system on object selection tasks and describe our initial efforts to integrate this system into a multimodal conversational dialog framework.

This technical report provides a brief overview of how human hand gestures can be used in wearable Human Computer Interfaces (HCI).

Humans use a combination of gesture and speech to interact with objects and usually do so more naturally without holding a device or pointer. We present a system that incorporates user body-pose estimation, gesture recognition and speech recognition for interaction in virtual reality environments. We describe a vision-based method for tracking the pose of a user in real time and introduce a technique that provides parameterized gesture recognition. More precisely, we train a support vector classifier to model the boundary of the space of possible gestures, and train Hidden Markov Models on specific gestures. Given a sequence, we can find the start and end of various gestures using a support vector classifier, and find gesture likelihoods and parameters with a HMM. A multimodal recognition process is performed using rank-order fusion to merge speech and vision hypotheses. Finally we describe the use of our multimodal framework in a virtual world application that allows users to interact using gestures and speech.

Navigating virtual environments usually requires a wired interface, game console, or keyboard. The advent of perceptual interface techniques allows a new option: the passive and untethered sensing of users' pose and gesture to allow them maneuver through and manipulate virtual worlds. We describe new algorithms for interacting with 3-D environments using real-time articulated body tracking with standard cameras and personal computers. Our method is based on rigid stereo-motion estimation algorithms and uses a linear technique for enforcing articulation constraints. With our tracking system users can navigate virtual environments using 3-D gesture and body poses. We analyze the space of possible perceptual interface abstractions for full-body navigation, and present a prototype system based on these results. We finally describe an initial evaluation of our prototype system with users guiding avatars through a series of 3-D virtual game worlds.

Tabletop computer displays suffer from an orientation problem. Solutions based on competing approaches have been implemented, but there remain unanswered questions about the ideal solution. Within a context of interacting with documents on tabletop displays, requirements for orientation control were found by examining literature on how people use paper documents and what manipulations they perform. It was determined that control must provide quick, either-handed, low-attention manipulation of individual objects on the display. An evaluation of existing interaction techniques for rotation was performed in light of these criteria. The trade-off between two desirable characteristics, integral translation and rotation, and direct input, was examined. An evaluation of mouse-based techniques confirmed that integral input has the potential to be faster than the established sequential-manipulation techniques due to the time saved by overlapping manipulation of different degrees of freedom. For a single task combining translation and rotation the separable technique took twice as long compared to two separate tasks of the same translation or rotation, whereas the integral techniques took less time for the combined action than the two separate actions. However, the integral mouse-based techniques were too slow in their actual manipulation, with the scroll wheel taking four times as long to rotate than the separable technique, and the new drag technique not being used in an integral manner. It was concluded that the current generally available technology such as the mouse and single-point touch-sensitive overlays are inadequate. Acceptable tabletop orientation control will be dependent on the maturation of newer technologies based on tangible interfaces or possibly multi-finger input. Until then, interaction techniques for manipulating documents on tabletop displays should be chosen according to their ability to concurrently control position and orientation, such as three degree-of-freedom input devices on digitizing tablets.

In this paper, we discuss a new hand-gesture-based application, FlowField, which explores the use of pressure-sensitive multipoint touchpad (multi-touch) interaction. FlowField allows participants to interact using their whole hand with a flow of circulating particles, providing visual and auditory feedback. Limitations of the raw data mapping used in FlowField motivated work on applying interpolation techniques to improve the data representation. Thus far, bicubic interpolation provides the most effective method for initial processing. We observed two factors limiting the quality of the representation: the spacing of the sensors, and the pressure-distributing property of the touchpad surface. 1.

This paper describes SmartCanvas, an intelligent desk system that allows a user to perform freehand drawing on a desk or similar surface with gestures. Our system requires one camera and no touch sensors. The key underlying technique is a vision-based method that distinguishes drawing gestures and transitional gestures in real time, avoiding the need for "artificial" gestures to mark the beginning and end of a drawing stroke. The method achieves an average classification accuracy of 92.17%. Pie-shaped menus and a "rotateto -and-select" approach eliminate the need for a fixed menu display, resulting in an "invisible" interface.