We introduce a novel method to diagnose pronunciation errors that are most critical to the intelligibility of L2 learners. A preliminary study showed that error rates computed by a speech recognition-based system can be used to characterize intelligibility. We deduce a probabilistic algorithm to derive intelligibility from error rates. We also define an error priority function that indicates which errors are most critical to intelligibility. Experimental results proved the validity of the approach.

This paper investigates the feasibility of automated scoring of spoken English proficiency of non-native speakers. Unlike existing automated assessments of spoken English, our data consists of spontaneous spoken responses to complex test items. We first compute a set of features relevant for measuring communicative competence based on speech recognition output. We then perform both a quantitative and a qualitative analysis of these features using two different machine learning approaches. (1) We use support vector machines to produce a score and evaluate it with respect to a mode baseline and to human rater agreement. We find that scoring based on support vector machines yields accuracies approaching inter-rater agreement in some cases. (2) We use classification and regression trees to understand the role of different features and feature classes in the characterization of speaking proficiency by human scorers. Our analysis shows that across all the test items most or all the feature classes are used in the nodes of the trees suggesting that the scores are, appropriately, a combination of multiple components of speaking proficiency. Future research will concentrate on extending the set of features and introducing new feature classes to arrive at a scoring model that comprises additional relevant aspects of speaking proficiency.

We introduce a novel method to diagnose pronunciation errors that are most critical to the intelligibility of L2 learners. A prelimiary study showed that error rates computed by a speech recognition-based system can be used to characterize intelligibility. We deduce a probabilistic algorithm to derive intelligibility from error rates. We also define an error priority function that indicates which errors are most critical to intelligibility. Experimental results proved the validity of the approach.

Language Students ’ Pronunciation?

This paper focuses on identifying, extracting and evaluating features related to syntactic complexity of spontaneous spoken responses as part of an effort to expand the current feature set of an automated speech scoring system in order to cover additional aspects considered important in the construct of communicative competence. Our goal is to find effective features, selected from a large set of features proposed previously and some new features designed in analogous ways from a syntactic complexity perspective that correlate well with human ratings of the same spoken responses, and to build automatic scoring models based on the most promising features by using machine learning methods. On human transcriptions with manually annotated clause and sentence boundaries, our best scoring model achieves an overall Pearson correlation with human rater scores of r=0.49 on an unseen test set, whereas correlations of models using sentence or clause boundaries from automated classifiers are around r=0.2. 1

This paper describes a system aimed at automatically scoring two task types of high and medium-high linguistic entropy from a spoken English test with a total of six widely differing task types. We describe the speech recognizer used for this system and its acoustic model and language model adaptation; the speech features computed based on the recognition output; and finally the scoring models based on multiple regression and classification trees. For both tasks, agreement measures between machine and human scores (correlation, kappa) are close to or reach inter-human agreements. 1

Abstract not found