This paper shows that automatically generated questions can help classify children’s spoken responses to a reading tutor teaching them to generate their own questions. We use automatic question generation to model and classify children’s prompted spoken questions about stories. On distinguishing complete and incomplete questions from irrelevant speech and silence, a language model built from automatically generated questions out-performs a trigram language model that does not exploit the structure of questions.

This paper focuses on the analysis and prediction of student disengagement and uncertainty, using a corpus of dialogues collected with a spoken tutorial dialogue system in the STEM domain of qualitative physics. We first compare and contrast the prosodic characteristics of dialogue turns exhibiting disengagement or not, and those exhibiting uncertainty or not. We then compare the utility of using multiple prosodic features to predict both disengagement and uncertainty. Index Terms: spoken dialogue systems, educational applications, emotion detection, prosody

Abstract 1 The accuracy of an automated reading tutor in tracking the reader’s position is affected by phenomena at the frontier of the speech recognizer’s output as it evolves in real time. We define metrics of real-time tracking accuracy computed from the recognizer’s successive partial hypotheses, in contrast to previous metrics computed from the final hypothesis. We analyze the resulting considerable loss in real-time accuracy, and propose and evaluate a method to address it. Our method raises real-time accuracy from 58 % to 70%, which should improve the quality of the tutor’s feedback.

This paper introduces some of the research behind automatic scoring of the speak-ing part of the Arizona English Language Learner Assessment, a large-scale test now operational for students in Arizona. Ap-proximately 70 % of the students tested are in the range 4-11 years old. We cover the methods used to assess spoken responses automatically, considering both what the student says and the way in which the stu-dent speaks. We also provide evidence for the validity of machine scores. The assessments include 10 open-ended item types. For 9 of the 10 open item types, machine scoring performed at a similar level or better than human scoring at the item-type level. At the participant level, correlation coefficients between machine overall scores and average human overall

This work introduces new methods for de-tecting non-scorable tests, i.e., tests that cannot be accurately scored automatically, in educational applications of spoken lan-guage proficiency assessment. Those in-clude cases of unreliable automatic speech recognition (ASR), often because of noisy, off-topic, foreign or unintelligible speech. We examine features that estimate signal-derived syllable information and compare it with ASR results in order to detect responses with problematic recognition. Further, we explore the usefulness of lan-guage model based features, both for lan-guage models that are highly constrained to the spoken task, and for task inde-pendent phoneme language models. We validate our methods on a challenging dataset of young English language learn-ers (ELLs) interacting with an automatic spoken assessment system. Our proposed methods achieve comparable performance compared to existing non-scorable detec-tion approaches, and lead to a 21 % rela-tive performance increase when combined with existing approaches. 1