Abstract. This study proposes a roadmap for the creation and specification of a virtual humanoid capable of performing expressive gestures in real time. We present a gesture motion data acquisition protocol capable of handling the main articulators involved in human expressive gesture (whole body, fingers and face). The focus is then shifted to the postprocessing of captured data leading to a motion database complying with our motion specification language and capable of feeding data driven animation techniques. Issues. Embodying a virtual humanoid with expressive gestures raises many problems such as computation-cost efficiency, realism and level of expressiveness, or high level specification of expressive gesture [1]. Here, we focus on the acquisition of motion capture data from the main articulators involved in communicative gesture (whole body, face mimics and finger motion). We then show how acquired data are postprocessed in order to build a database compatible with high level gesture specification and capable of feeding real time data-driven

Temporal segmentation of human motion into actions is central to the understanding and building of computational models of human motion and activity recognition. Several issues contribute to the challenge of temporal segmentation and classification of human motion. These include the large variability in the temporal scale and periodicity of human actions, the complexity of representing articulated motion, and the exponential nature of all possible movement combinations. We provide initial results from investigating two distinct problems- classification of the overall task being performed, and the more difficult problem of classifying individual frames over time into specific actions. We explore first-person sensing through a wearable camera and Inertial Measurement Units (IMUs) for temporally segmenting human motion into actions and performing activity classification in the context of cooking and recipe preparation in a natural environment. We present baseline results for supervised and unsupervised temporal segmentation, and recipe recognition in the CMU-Multimodal activity database (CMU-MMAC).

The problem addressed in this paper concerns the representation of human movement in terms of atomic visuo-motor primitives considering both generation and perception of movement. We introduce the concept of kinetology, the phonology of human movement, and five principles on which such a system should be based: compactness, view-invariance, reproducibility, selectivity, and reconstructivity. We propose visuo-motor primitives and demonstrate their kinetological properties. Further evaluation is accomplished with experiments on compression and decompression. Our long-term goal is to demonstrate that action has a space characterized by a visuo-motor language.

We synchronize background music with an animation by changing the timing of both, an approach which minimizes the damage to either. Starting from a MIDI file and motion data, feature points are extracted from both sources, paired, and then synchronized using dynamic programming to time-scale the music and to timewarp the motion. We also introduce the music graph, a directed graph which encapsulates connections between many short music sequences. By traversing a music graph we can generate large amounts of new background music, in which we expect to find a sequence which matches the motion better than the original music. Categories and Subject Descriptors (according to ACM CCS): J.5 [Computer Applications]: Arts And Humanities 1.

Abstract not found

Imitation learning is a promising technique for teaching robots complex movement sequences. One key problem in this area is the transfer of perceived movement characteristics from perception to action. For the solution of this problem, representations are required that are suitable for the analysis and the synthesis of complex action sequences. We describe the method of Hierarchical Spatio-Temporal Morphable Models that allows an automatic segmentation of movements sequences into movement primitives, and a modeling of these primitives by morphing between a set of prototypical trajectories. We use HSTMMs in an imitation learning task for human writing movements. The models are learned from recorded trajectories and transferred to a human-like robot arm. Due to the generalization properties of our representation, the arm is capable of synthesizing new writing movements with a few learning examples. 1

“Which is also... one of those very funny episodes... that are in... this movie.” Figure 1: Data-driven body language is synthesized from live speech input. Human communication involves not only speech, but also a wide variety of gestures and body motions. Interactions in virtual environments often lack this multi-modal aspect of communication. We present a method for automatically synthesizing body language animations directly from the participants ’ speech signals, without the need for additional input. Our system generates appropriate body language animations by selecting segments from motion capture data of real people in conversation. The synthesis can be performed progressively, with no advance knowledge of the utterance, making the system suitable for animating characters from live human speech. The selection is driven by a hidden Markov model and uses prosody-based features extracted from speech. The training phase is fully automatic and does not require hand-labeling of input data, and the synthesis phase is efficient enough to run in real time on live microphone input. User studies confirm that our method is able to produce realistic and compelling body language.

Humanoid robots will increasingly become a part of human everyday lives, but require more natural and simplified methods for control and humanrobot interaction. Our approach to addressing these challenges is to use biologically inspired notions of behavior-based control, and endow robots with the ability to imitate, so that they can be programmed and interacted with through demonstration and imitation. Our approach to the problem, based on neuroscience evidence, structures the motor system into a collection of primitives, which are then used to both generate the humanoid 's movement repertoire and provide prediction and classification capabilities for visual perception and interpretation. Thus, what the humanoid can do helps it understand what it sees, and vice versa. We describe the behavior-based background of our work and the neuroscience evidence on which our humanoid motor control and imitation model is based. Next we describe our use of human movement data as ...

Motion capture data often requires substantial processing before it becomes useful. We propose a technique that automatically distills a compact motion graph from an arbitrary collection of motion capture data. At its heart, the process identifies clusters of similar motions which we call “motion bundles”. Motion bundles and their encompassing motion graph provide a readily understandable structuring of the motion data. They can serve as a shared tool in support of common types of motion processing, including motion segmentation, motion compression, the creation of blend spaces, and the identification of connectivity to support motion resequencing. We use a novel string-based representation of motions to help find motion bundles. Users can specify a preference for longduration bundles or bundles containing many motions. We demonstrate results using data for boxing, walking and various exercise motions, and we show that meaningful partitions are retained in the face of noise.

Abstract — Researchers and engineers have used primitive actions to facilitate programming of tasks since the days of Shakey [1]. Task-level programming, which requires the user to specify only subgoals of a task to be accomplished, depends on such a set of primitive task programs to perform these subgoals. Past research in this area has used the commands from robot programming languages as the vocabulary of primitive tasks for robotic manipulators. We propose drawing from work measurement systems to construct the vocabulary of primitive task programs. We describe one such work measurement system, present several primitive task programs for humanoid robots inspired from this system, and show how these primitive programs can be used to construct complex behaviors. Index Terms — robot programming, task-level programming, humanoid robots I.