We have built a system that allows users to naturally manipulate virtual 3D models with both hands on the Responsive Workbench, a tabletop VR device. Our design is largely based upon Guiard's observations of how humans distribute work between the two hands in the real world. We show how to apply these principles for the workbenchenvironment and describe many issues encountered during the design. We first develop a framework for two-handed interaction and then explore a variety of two-handed 3D tools and interactive techniques. Related issues include how constraints are implemented and controlled by the two hands and how transitions between one-handedandtwo-handed tasks occur seemlessly. Informal observations of the system in practice show that users can perform navigation and manipulation tasks easily and with little training using the two-handed environment. One of our interesting findings was that users often performed two-handed manipulations by combining two otherwise independent one-handed tools in a synergistic fashion. In these cases, we did not program two-handed behaviors explicitly into the system; instead they emerged naturally.

An experimental GUI paradigm is presented which is based on the design goals of maximizing the amount of screen used for application data, reducing the amount that the UI diverts visual attentions from the application data, and increasing the quality of input. In pursuit of these goals, we integrated the non-standard UI technologies of multi-sensor tablets, toolglass, transparent UI components, and marking menus. We describe a working prototype of our new paradigm, the rationale behind it and our experiences introducing it into an existing application. Finally, we presents some of the lessons learned: prototypes are useful to break the barriers imposed by conventional GUI design and some of their ideas can still be retrofitted seamlessly into products. Furthermore, the added functionality is not measured only in terms of user performance, but also by the quality of interaction, which allows artists to create new graphic vocabularies and graphic styles. KEYWORDS: two-handed input, toolglass, tablets, transparency, marking menus, task integration, divided attention

We present an experiment on cooperative bimanual action. Right-handed subjects manipulated a pair of physical objects, a tool and a target object, so that the tool would touch a target on the object @g. 1). For this task, there is a marked specialization of the hands. Performance is best when the left hand orients the target object and the right hand manipulates the tool, but is significantly reduced when these roles are reversed. This suggests that the right hand operates relative to the frame-of-reference of the left hand. Furthermore, when physical constraints guide the tool placement, this fundamentally changes the type of motor control required. The task is tremendously simplified for both hands, and reversing roles of the hands is no longer an important factor. Thus, specialization of the roles of the hands is significant only for skilled manipulation.

Manipulation in immersive virtual environments is difficult partly because users must do without the haptic contact with real objects they rely on in the real world to orient themselves and the objects they are manipulating. To compensate for this lack, I propose exploiting the one real object every user has in a virtual environment, his body. I present a unified framework for virtual-environment interaction based on proprioception, a person's sense of the position and orientation of his body and limbs. I describe three forms of body-relative interaction: . Direct manipulation---ways to use body sense to help control manipulation . Physical mnemonics---ways to store/recall information relative to the body . Gestural actions---ways to use body-relative actions to issue commands Automatic scaling is a way to bring objects instantly within reach so that users can manipulate them using proprioceptive cues. Several novel virtual interaction techniques based upon automatic scaling and our proposed framework of proprioception allow a user to interact with a virtual world intuitively, efficiently, precisely, and lazily. Two formal user studies evaluate key aspects of body-relative interaction. The virtual docking study compares the manipulation of objects co-located with one's hand and the manipulation of objects at a distance. The widget interaction experiment explores the differences between interacting with a widget held in one's hand and interacting with a widget floating in space. Lessons learned from the integration of body-relative techniques into a real-world system, the Chapel Hill Immersive Modeling Program (CHIMP), are presented and discus...