Abstract

We present a new parser for parsing down to Penn tree-bank style parse trees that achieves 90.1% average precision/recall for sentences of length 40 and less, and 89.5% for sentences of length 100 and less when trained and tested on the previously established [5,9,10,15,17] \standard " sections of the Wall Street Journal treebank. This represents a 13% decrease in error rate over the best single-parser results on this corpus [9]. The major technical innovation is the use of a \maximum-entropy-inspired" model for conditioning and smoothing that let us successfully to test and combine many dierent conditioning events. We also present some partial results showing the eects of dierent conditioning information, including a surprising 2% improvement due to guessing the lexical head's pre-terminal before guessing the lexical head. 1 Introduction We present a new parser for parsing down to Penn tree-bank style parse trees [16] that achieves 90.1% average precision/recall for sentences of ...

Citations

1654	B.: Building a large annotated corpus of english: The Penn treebank - Marcus, Marcinkiewicz, et al. - 1993
846	A Maximum Entropy Approach to Natural Language Processing - Berger, Pietra, et al. - 1996
780	Head-Driven Statistical Models for Natural Language Parsing - Collins - 1999
649	A stochastic parts program and noun phrase parser for unrestricted text - Church - 1988
496	Statistical Language Learning - Charniak - 1993
433	A simple rule-based part of speech tagger - Brill - 1992
427	Three Generative, Lexicalised Models for Statistical Parsing - Collins - 1997
396	New Statistical Parser Based on Bigram Lexical De-pendencies - Collins - 2006
355	Generalized Iterative Scaling for Log-Linear Models - Darroch, Ratcliff - 1972
324	Statistical Parsing with a Context-Free Grammar and Word Statistics - Charniak - 1997
287	Statistical Decision-Tree Models for Parsing - Magerman - 1995
259	Frequency analysis of English usage: Lexicon and grammar - Francis, Kucera - 1982
227	Some advances in transformation-based part of speech tagging - Brill
203	Tree-Bank Grammars - Charniak - 1996
174	PCFG Models of Linguistic Tree Representations’. Computational Linguistics 24:613–632 - Johnson - 1998
148	Grammatical category disambiguation by statistical optimization - DeRose - 1988
136	Learning to Parse Natural Language with Maximum Entropy Models’. Machine Learning 34:151–175 - Ratnaparkhi - 1999
126	Coping with Ambiguity and Unknown Words through Probabilistic Models - Weischedel, Meteer, et al. - 1993
98	Equations for part-of-speech tagging - Charniak, Hendrickson, et al. - 1993
78	Statistical techniques for natural language parsing - Charniak - 1997
65	New figures of merit for best first probabilistic chart parsing - Caraballo, Charniak - 1997
63	Parsing the LOB corpus - Marcken - 1990
53	Exploiting diversity in natural language processing - Henderson, Brill - 1999
49	Markov source modeling of text generation - Jelinek - 1985
45	Edge-based best-first chart parsing - Charniak, Goldwater, et al. - 1998
40	Context-sensitive statistics for improved grammatical language models - Systems, Charniak, et al. - 1994
37	An empirical comparison of probability models for dependency grammar - Eisner - 1996
23	Disambiguation of prepositional phrases in automatically labelled technical text - Boggess, Agarwal, et al. - 1991
9	Expected-frequency interpolation - Charniak - 1996
9	Training stochastic grammars from unlabelled text corpora - Kupiec, Maxwell - 1992
8	New of merit for best- probabilistic chart parsing - Caraballo, Charniak - 1998
4	Shipping departments vs. shipping pacemakers: using thematic analysis to improve tagging accuracy - Zernik - 1992
2	Edge-based best- chart parsing - Charniak, Goldwater, et al. - 1998