Modern wearable computer designs package workstation level performance in systems small enough to be worn as clothing. These machines enable technology to be brought where it is needed the most for the handicapped: everyday mobile environments. This paper de- scribes a research effort to make a wearable computer that can recognize (with the possible goal of translat- ing) sentence level American Sign Language (ASL) using only a baseball cap mounted camera for input. Current accuracy exceeds 97% per word on a 40 word lexicon.

Instrumented gloves -- gloves equipped with sensors for detecting finger bend, hand position and orientation -- were conceived to allow a more natural interface to computers. However, the extension of their use for recognising sign language, and in this case Auslan (Australian Sign Language), is possible. Several researchers have already explored these possibilities and have successfully achieved finger-spelling recognition with high levels of accuracy, but progress in the recognition of sign language as a whole has been limited.

We build upon a constrained, lab-based Sign Language recognition system with the goal of making it a mobile assistive technology. We examine using multiple sensors for disambiguation of noisy data to improve recognition accuracy. Our experiment compares the results of training a small gesture vocabulary using noisy vision data, accelerometer data and both data sets combined.

Humans use gestures in most communicative acts. How are these gestures initiated and performed? What kinds of communicative roles do they play and what kinds of meanings do they convey? How do listeners extract and understand these meanings? Will it be possible to build computerized communicating agents that can extract and understand the meanings and accordingly simulate and display expressive gestures on the computer in such a way that they can be effective conversational partners? All these questions are easy to ask, but far more difficult to answer. In the thesis we try to address these questions regarding the synthesis and acquisition of communicative gestures. Our approach to gesture is...

A divide-and-conquer approach is presented for signer-independent continuous Chinese Sign Language(CSL) recognition in this paper. The problem of continuous CSL recognition is divided into the subproblems of isolated CSL recognition. We combine the simple recurrent network(SRN) with the hidden Markov models(HMM) in this approach. The improved SRN is introduced for segmentation of continuous CSL. Outputs of SRN are regarded as the states of HMM, and the Lattice Viterbi algorithm is employed to search the best word sequence in the HMM framework. Experimental results show SRN/HMM approach has better performance than the standard HMM one.

Emergency response requires strategic assessment of risks, decisions, and communications that are timecritical while requiring teams of individuals to have fast access to large volumes of complex information and technologies that enables tightly coordinated work. The access to this information by crisis management (CM) teams in emergency operations centers can be facilitated through various humancomputer interfaces. Unfortunately these interfaces are hard to use, require extensive training, and often impede rather than support teamwork. Dialogue-enabled devices, based on natural, multimodal interfaces have the potential of making a variety of information technology tools accessible during crisis management. This paper establishes the importance of multimodal interfaces in various aspects of crisis management and explores many issues in realizing successful speech-gesture driven, dialog-enabled interfaces for CM. The paper

Instrumented gloves use a variety of sensors to provide information about the user's hand. They can be used for recognition of gestures; especially well-defined gesture sets such as sign languages. However, recognising gestures is a difficult task, due to intrapersonal and interpersonal variations in performing them. One approach to solving this problem is to use machine learning. In this case, samples of 95 discrete Australian Sign Language (Auslan) signs were collected using a PowerGlove. Two machine learning techniques were applied -- instance-based learning (IBL) and decision-tree learning -- to the data after some simple features were extracted. Accuracy of approximately 80 per cent was achieved using IBL, despite the severe limitations of the glove. Introduction Sign language recognition is interesting for a number reasons; it represents an interesting domain in itself with obvious real-world applications, but it also makes a good starting point for gesture recognition in genera...

Abstract not found

Recent approaches to providing users with a more natural method of interacting with computer applications have shown that more than one mode of input can be both beneficial and intuitive as a communication medium between humans and computers. Two modalities in particular, whole-hand and speech input, represent a natural form of communication that has been ingrained in our physical and mental makeup since birth. In this thesis, we investigate the use of whole-hand and speech input in virtual environments in the context of two applications domains: scientific visualization and interior design. By examining the two modalities individually and in combination, and through the creation of two application prototypes (Multimodal Scientific Visualization Tool and Room Designer), we present anumber of contributions including a set of interface guidelines and interaction techniques for whole-hand and speech input.

Abstract: For the past two years we have been addressing the challenges involved in managing the data generated within immersive environments. We together with many other researchers have addressed the management of obvious data types such as image, video, audio and text. However, we identified a set of less familiar data types, collectively termed as immersidata, that are specific to immersive environments. In this paper, we focus our attention on analysis of a kind of immersidata known as haptic data. We propose to analyze the haptic data acquired from CyberGlove to recognize different static hand signs automatically. The ultimate objective is to understand how to model and store haptic data in a database, for similar types of applications. We propose several techniques to analyze subtle changes in hand signs and words (a series of signs). We show that our techniques can recognize the most important features to distinguish between two letters and several preliminary experiments demonstrate more than 84.66 % accuracy in sign recognition for a 10-sign vocabulary. 1