We integrate automatic speech recognition (ASR) and question answering (QA) to realize a speech-driven QA system, and evaluate its performance. We adapt an Ngram language model to natural language questions, so that the input of our system can be recognized with a high accuracy. We target WH-questions which consist of the topic part and fixed phrase used to ask about something. We first produce a general N-gram model intended to recognize the topic and emphasize the counts of the N-grams that correspond to the fixed phrases. Given a transcription by the ASR engine, the QA engine extracts the answer candidates from target documents. We propose a passage retrieval method robust against recognition errors in the transcription. We use the QA test collection produced in NTCIR, which is a TREC-style evaluation workshop, and show the effectiveness of our method by means of experiments. 1.

Abstract. In this paper, we propose a new and valuable research task: opendomain question answering system with speech interface, and first prototype (SpeechQoogle) is constructed with three separated modules: speech recognition, question answering (QA) and speech synthesis. Speech interface improves the utility of QA system, but also brings several new challenges, including 1) distorting effect of speech recognition error; 2) just one answer could be returned; and 3) the returned answer should be understandable just in speech. To conquer these challenges in SpeechQoogle’s construction, we first carefully choose FAQ-based question answering technique for QA module because of its inherent advances and 600,000 QA pairs are collected to support this technique. Then corresponding acoustic model and language model are particularly developed for speech recognition module which promotes the character ACC to 87.17%. Finally, in open-set testing the integrated prototype successfully answers 56.25 % spoken questions, which is a quite satisfied and inspiring performance because many potential improving approaches are still unexploited.