. Passive sensing of human hand and limb motion is important for a wide range of applications from human-computer interaction to athletic performance measurement. High degree of freedom articulated mechanisms like the human hand are difficult to track because of their large state space and complex image appearance. This article describes a model-based hand tracking system, called DigitEyes, that can recover the state of a 27 DOF hand model from ordinary gray scale images at speeds of up to 10 Hz. 1 Introduction Sensing of human hand and limb motion is important in applications from Human-Computer Interaction (HCI) to athletic performance measurement. Current commercially available solutions are invasive, and require the user to don gloves [15] or wear targets [8]. This paper describes a noninvasive visual hand tracking system, called DigitEyes. We have demonstrated hand tracking at speeds of up to 10 Hz using line and point features extracted from gray scale images of unadorne...

Passive sensing of human hand and limb motion is important for a wide range of applications from human-computer interaction to athletic performance measurement. High degree of freedom articulated mechanisms like the human hand are difficult to track because of their large state space and complex image appearance. This article describes a model-based hand tracking system, called DigitEyes, that can recover the state of a 27 DOF hand model from gray scale images at speeds of up to 10 Hz. We employ kinematic and geometric hand models, along with a high temporal sampling rate, to decompose global image patterns into incremental, local motions of simple shapes. Hand pose and joint angles are estimated from line and point features extracted from images of unmarked, unadorned hands, taken from one or more viewpoints. We present some preliminary results on a 3D mouse interface based on the DigitEyes sensor. Contents 1 Introduction 2 2 The Articulated Mechanism Tracking Problem 2 3 State Mod...

A new approach is presented for the animation of articulated figures. We propose a system of articulated motion design which offers a full combination of both direct and inverse kinematic control of the joint parameters. Such an approach allows an animator to interactively specify goal-directed changes to existing sampled joint motions, resulting in a more general and expressive class of possible joint motions. The fundamental idea is to consider any desired joint space motion as a reference model inserted into the secondary task of an inverse kinematic control scheme. This approach profits from the use of halfspace cartesian main tasks in conjunction with a parallel control of the articulated figure called the coach-trainee metaphor.In addition, a transition function is introduced so as to guarantee the continuity of the control. The resulting combined kinematic control scheme leads to a new methodology of joint motion editing which is demonstrated through the improvement of a functional model of human walking.

Abstract not found

We present a realistic skeletal musculo-tendon model of the human hand and forearm. The model permits direct forward dynamics simulation, which accurately predicts hand and finger position given a set of muscle activations. We also present a solution to the inverse problem of determining an optimal set of muscle activations to achieve a given pose or motion; muscle fatigue, injury or atrophy can also be specified, yielding different control solutions that favour healthy muscle. As there can be many (or no) solutions to this inverse problem, we demonstrate how the space of possible solutions can be filtered to an optimal representative. Of particular note is the ability of our model to take a wide array of joint interdependence into account for both forward and inverse problems. Given kinematic postures, the model can be used to validate, predict or fill in missing motion and improve coarsely specified motion with anatomic fidelity. Lastly, we address the visualization and understanding of the dynamically changing and spatially compact musculature using various interaction techniques. 1.

Conventional methods for programming a robot either are inflexible or demand significant expertise. While the notion of automatic programming by high-level goal specification addresses these issues, the overwhelming complexity of planning manipulator grasps and paths remains a formidable obstacle to practical implementation. This thesis describes the approach of programming a robot by human demonstration. Our system observes a human performing the task, recognizes the human grasp, and maps it onto the manipulator. Using human actions to guide robot execution greatly reduces the planning complexity. In analyzing the task sequence, the system first divides the observed sensory data into meaningful temporal segments, namely the pregrasp, grasp, and manipulation phases. This is achieved by analyzing the human hand motion profiles. The features used are the fingertip polygon area (the fingertip polygon being the polygon whose vertices are the fingertips), hand speed, and the volume sweep r...

Simulating a human figure performing a task requires that the agent interact with objects in the environment in a realistic manner. In this paper we describe a system which directs task-level, general-purpose, object grasping for a simulated human agent. The Object Specific Reasoner (OSR) generates parameters for underspecified task-level instructions such as (pickup jack hammer). The Grasp behavior manages simultaneous motions of the joints in the hand, wrist and arm. When composed hierarchically, the OSR and the Grasp behavior interpret task-level commands to the animation system. These modules are implemented as part of the Jack project at the University of Pennsylvania.

In this paper, we present current research developments in the Virtual Life of autonomous synthetic actors. After a brief description of the perception action principles with a few simple examples, we emphasize the concept of virtual sensors for virtual humans. In particular, we describe in details our experiences in implementing virtual vision, tactile and audition. We then describe perception-based locomotion, a multisensor based method of automatic grasping and vision-based ball games.

Three-dimensional (3D) kinematic models are widely-used in videobased figure tracking. We show that these models can suffer from singularities when motion is directed along the viewing axis of a single camera. The single camera case is important because it arises in many interesting applications, such as motion capture from movie footage, video surveillance, and vision-based user-interfaces. We describe

Abstract — A system is detailed here for using imitation learning to teach a robot to grasp objects using both hand and wholebody grasps, which use the arms and torso as well as hands. Demonstration grasp trajectories are created by teleoperating a simulated robot to pick up simulated objects, modeled as combinations of up to three aligned primitives—boxes, cylinders, and spheres. When presented with a target object, the system compares it against the objects in a stored database to pick a demonstrated grasp used on a similar object. By considering the target object to be a transformed version of the demonstration object, contact points are mapped from one object to the other. The most promising grasp candidate is chosen with the aid of a grasp quality metric. To test the success of the chosen grasp, a collision-free grasp trajectory is found and an attempt is made to execute it in simulation. The implemented system successfully picks up 92 out of 100 randomly generated test objects in simulation. I.