Figure 1: We reconstruct a stationary sleeve using thousands of markers to estimate the geometry (texture added with bump mapping). We capture the shape of moving cloth using a custom set of color markers printed on the surface of the cloth. The output is a sequence of triangle meshes with static connectivity and with detail at the scale of individual markers in both smooth and folded regions. We compute markers ’ coordinates in space using correspondence across multiple synchronized video cameras. Correspondence is determined from color information in small neighborhoods and refined using a novel strain pruning process. Final correspondence does not require neighborhood information. We use a novel data driven hole-filling technique to fill occluded regions. Our results include several challenging examples: a wrinkled shirt sleeve, a dancing pair of pants, and a rag tossed onto a cup. Finally, we demonstrate that cloth capture is reusable by animating a pair of pants using human motion capture data. 1

Figure 1: Left to right: an actor performing in the capture setup; one of sixteen views from the camera array; reconstructed T-shirt geometry; the real T-shirt is replaced by a rendering of the captured geometry with different appearance characteristics. A lot of research has recently focused on the problem of capturing the geometry and motion of garments. Such work usually relies on special markers printed on the fabric to establish temporally coherent correspondences between points on the garment’s surface at different times. Unfortunately, this approach is tedious and prevents the capture of off-the-shelf clothing made from interesting fabrics. In this paper, we describe a marker-free approach to capturing garment motion that avoids these downsides. We establish temporally coherent parameterizations between incomplete geometries that we extract at each timestep with a multiview stereo algorithm. We then fill holes in the geometry using a template. This approach, for the first time, allows us to capture the geometry and motion of unpatterned, off-the-shelf garments made from a range of different fabrics.

Abstract. We present a dynamic near-regular texture (NRT) tracking algorithm nested in a lattice-based Markov-Random-Field (MRF) model of a 3D spatiotemporal space. One basic observation used in our work is that the lattice structure of a dynamic NRT remains invariant despite its drastic geometry or appearance variations. On the other hand, dynamic NRT imposes special computational challenges to the state of the art tracking algorithms: including highly ambiguous correspondences, occlusions, and drastic illumination and appearance variations. Our tracking algorithm takes advantage of the topological invariant property of the dynamic NRT by combining a global lattice structure that characterizes the topological constraint among multiple textons and an image observation model that handles local geometry and appearance variations. Without any assumptions on the types of motion, camera model or lighting conditions, our tracking algorithm can effectively capture the varying underlying lattice structure of a dynamic NRT in different real world examples, including moving cloth, underwater patterns and marching crowd. 1

Recent years have seen an increased interest in cloth simulation. There has been little analysis, however, of the parameters controlling simulation behaviour. In this thesis, we present two primary contributions. First, we discuss a series of experiments investigating the influence of the parameters of a popular cloth simulation algorithm. Second, we present a system for motion capture of deformable surfaces, most notably moving cloth, including both geometry and parameterisation. This data could subsequently be used for the recovery of cloth simulator parameters. In our motion capture system, we recover geometry using stereo correspondence, and use the Scale Invariant Feature Transform (SIFT) to identify an arbitrary pattern printed on the cloth, even in the presence of fast motion. We describe a novel seedand -grow approach to adapt the SIFT algorithm to deformable geometry. Finally, we interpolate feature points to parameterise the complete geometry.

Augmented reality (AR) is the concept of inserting virtual objects into real scenes. Typically, these augmentations are aligned with rigid planar objects in the scene, which can sometimes be restrictive. This poster presents a method to perform real-time 2D augmentations on nonrigid objects, such as clothing. In addition, a novel technique to establish common illumination and render augmentations with correct real world shadows is included. The ability to render augmentations on clothing leads to applications in fashion, advertising and museum entertainment. Results of the augmented clothing system are demonstrated with an application to interactively create t-shirt designs by augmenting logos. 1 System Overview This system operates on video images from a live camera. A set of trackable markers are placed on the surface of a piece of cloth. The marker locations are used to build a virtual mesh representation of the cloth that is rendered with a texture on top of the video image in each frame. In addition, the real world illumination environment is acquired from the input image and a shadow texture is generated and blended with the augmentation during rendering. The output is a mixed-reality video frame that contains an illumination-correct 2D augmentation on the surface of clothing. Figure 1 illustrates the operation of the system for each video frame. Figure 1: Overview of augmented clothing system. A similar cloth tracking method is proposed by Guskov et al. [2, 1]. However their system is designed for motion capture and recovery of the 3D model, not real-time augmentations with correct illumination.

personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Cloth is difficult to simulate because it forms small, complex folds. These folds make cloth configuration difficult to measure. Particular problems include fast motion (ruling out laser ranging methods), the necessity for high resolution measurement, the fact that no viewing direction can see into the folds, and the fact that many points are visible with either small baseline or in only one view. We describe a method that can recover high resolution measurements of the shape of real cloth. Our method uses multiple cameras, a special pattern printed on the cloth, and high shutter speeds

personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission.

We present an image-based algorithm for surface reconstruction of moving garments from multiple calibrated video cameras. Using a color-coded cloth texture, we reliably match circular features between different camera views. As surface model we use an a priori known triangle mesh. By identifying the mesh vertices with texture elements we obtain a coherent parametrization of the surface over time without further processing. Missing data points resulting from self-shadowing are plausibly interpolated by minimizing a thin-plate functional. The deforming geometry can be used for different graphics applications, e.g. for realistic retexturing. We show results for real garments demonstrating the accuracy of the recovered flexible shape.

We present a new image-based algorithm for surface reconstruction of moving garment from multiple calibrated video cameras. Using a color-coded cloth texture, we reliably match circular features between different camera views. As surface model we use an a priori known triangle mesh. By identifying the mesh vertices with texture elements we obtain a coherent parameterization of the surface over time without further processing. Missing data points resulting from self-shadowing are plausibly interpolated by minimizing a thin-plate functional. The deforming geometry can be used for different graphics applications, e.g. for realistic retexturing. We show results for real garments demonstrating the accuracy of the recovered flexible shape.

Articulated hand-tracking systems have been widely used in virtual reality but are rarely deployed in consumer applications due to their price and complexity. In this paper, we propose an easy-to-use and inexpensive system that facilitates 3-D articulated user-input using the hands. Our approach uses a single camera to track a hand wearing an ordinary cloth glove that is imprinted with a custom pattern. The pattern is designed to simplify the pose estimation problem, allowing us to employ a nearest-neighbor approach to track hands at interactive rates. We describe several proof-of-concept applications enabled by our system that we hope will provide a foundation for new interactions in modeling, animation control and augmented reality.