Many human behaviors re ect the attunement of our perceptual systems to rhythmic patterns of stimulation. Examples include dancing to music, speech communication, and the performance of a symphony orchestra. However, developing a computational model of rhythm perception has proven to be di cult for two main reasons. First, rhythm is holistic, yet rhythmic patterns evolve over time. Second, periodicities in rhythmic patterns typically exhibit variability in their timing. Many previous approaches to rhythm perception have ignored these two problems by abstracting time to the level of musical notation, and thus failed to address the fundamental issue of the perception of time. The approach taken in this thesis is that the developmentof a model of rhythm perception must rst address the perception of the time intervals which comprise rhythmic patterns. I propose a class of adaptive-oscillator processing units which track periodicities in rhythmic patterns (beats). Modest random variations in the timing of rhythmic patterns do not reduce the adaptive oscillator's ability to attain synchrony, and can even improve it. An Entrainment Model of human time perception is then developed. The

. Led by the fundamental role that rhythms apparently play in speech and gestural communication among humans, this study was undertaken to substantiate a biologically motivated model for synchronizing speech and gesture input in human computer interaction. Our approach presents a novel method which conceptualizes a multimodal user interface on the basis of timed agent systems. We use multiple agents for the purpose of polling presemantic information from different sensory channels (speech and hand gestures) and integrating them to multimodal data structures that can be processed by an application system which is again based on agent systems. This article motivates and presents technical work which exploits rhythmic patterns in the development of biologically and cognitively motivated mediator systems between humans and machines. 1 Introduction Gesture and speech are the corner stones in natural human communication. Not surprisingly, they are each paid considerable attentio...

It is proposed that the theory of dynamical systems offers appropriate tools to model many phonological aspects of both speech production and perception. A dynamic account of speech rhythm is shown to be useful for description of both Japanese mora timing and English timing in a phrase repetition task. This orientation contrasts fundamentally with the more familiar symbolic approach to phonology, in which time is modeled only with sequentially arrayed symbols. It is proposed that an adaptive oscillator offers a useful model for perceptual entrainment (or `locking in') to the temporal patterns of speech production. This helps to explain why speech is often perceived to be more regular than experimental measurements seem to justify. Because dynamic models deal with real time, they also help us understand how languages can differ in their temporal detail---contributing to foreign accents, for example. The fact that languages differ greatly in their temporal detail suggests that these effe...

The effect of deviations from temporal expectations on tempo discrimination was studied in 3 experiments using isochronous auditory sequences. Temporal deviations consisted of advancing or delaying the onset of a comparison pattern relative to an "expected " onset, defined by an extension of the periodicity of a preceding standard pattern. An effect of onset condition was most apparent when responses to faster and slower comparison patterns were analyzed separately and onset conditions were mixed. Under these conditions, early onsets produced more "faster " judgments and lower thresholds for tempo increases, and late onsets produced more "slower " judgments and lower thresholds for tempo decreases. In another experiment, pattern tempo had a similar effect: Fast tempos led to lower thresholds for tempo increases and slow tempos led to lower thresholds for tempo decreases. Findings support oscillator-based approaches to time discrimination. The perception and production of temporal patterns are fundamental abilities that are crucial for a wide range of human activity. Yet, there is still much that is not understood about some of the simplest temporal abilities. In particular,

An adaptive oscillator is a system that can lock on to a time-varying input signal, synchronizing its output to both the frequency and phase of the input. A wavetable oscillator generates a periodic output by indexing into a lookup table that stores a single period of the waveform. An adaptive wavetable oscillator (AWO) combines these two ideas in a technique which separates the periodic output waveform from the parameters that control the adaptation of the frequency and phase of the waveform. This separation is advantageous because it decouples the state of the oscillator from the dynamics of the adaptation, allowing the process of sychronization to be interpreted as a simple gradient optimization on a cost function. The oscillations remain stable over a large and easily described range of parameter values, and analysis of the synchronization can proceed along lines familiar from standard adaptive systems. Key issues in the design of AWOs are: the class of admissible inputs, the shape of the wavetable, the parameters that will be controlled, and the adaptive algorithm that adjusts the parameters. This paper examines these issues through analysis and simulation, focusing on conditions that achieve the desired synchronization between output and input.

This thesis examines an important issue in spoken word recognition; how the perceptual system segments connected speech into lexical units or words. Research on this topic has investigated the role of different sources of information in dividing up the speech stream: acoustic cues in the speech signal, statistical regularities in the structure of the language or through the identification of individual lexical items. This research focuses on cases in which the location of word boundaries may be ambiguous by one or more of these segmentation mechanisms using words embedded at the onset of longer words (such as cap in captain). The ambiguities proposed for onset-embedded words have motivated accounts of segmentation based on competition between alternative parses of speech into words. In these accounts, the recognition of embedded words is delayed until after their offset when subsequent input rules out longer competitors. In this thesis it is demonstrated that training a simple recurrent network to activate a representation of all the words in a sequence allows a connectionist network to learn the appropriate delay to allow the identification of onset-embedded words without requiring directly implemented competition between words. Both lexical competition and recurrent network models assume ambiguity between onset-embedded words and equivalent syllables in longer competitors. Acoustic analysis carried out in this thesis confirms the presence of reliable acoustic differences between syllables in short and long words. A series of experiments using gating and cross-modal priming suggest that the perceptual system uses these acoustic differences to discriminate embedded words from the onset of longer competitors and that match or mismatch with longer competitors may be less important for the identification of onset-embedded words. These results are interpreted within a revised version of the recurrent network model, incorporating input representing the acoustic differences between syllables in short and long words.

An adaptive oscillator is a system that can lock onto a timevarying input signal, synchronizing its output to both the frequency and phase of the input. A wavetable oscillator generates a periodic output by indexing into a lookup table that stores a single period of the waveform. An adaptive wavetable oscillator (AWO) combines these two ideas in a technique which separates the periodic output waveform from the parameters that control the adaptation of the frequency and phase of the waveform. The key issues in the design of AWOs are: the kind of oscillator to use, the class of admissible inputs, the shape of the wavetable, the control parameters, and the adaptive algorithm that adjusts the parameters. Wavetable oscillators can be applied to track the beat in MIDI signals, or, after an appropriate psycho-acoustical data reduction, to the tracking of audio signals. This paper examines these issues through analysis and simulation, focusing on conditions that achieve the desired entrainment between output and input. Sound examples demonstrate the application to beat tracking. Index Terms — adaptive systems, nonlinear oscillators, table lookup, oscillator stability 1.

this paper is to consider some general features of audition in higher animals, human or otherwise, and to describe several simulated components we have developed in our lab that we intend to serve as parts of an auditory system for such open environments. But what should such a system be able to do? Obviously that depends on many details about 2