Abstract

Discriminative methods have shown significant improvements over traditional generative methods in many machine learning applications, but there has been difficulty in extending them to natural language parsing. One problem is that much of the work on discriminative methods conflates changes to the learning method with changes to the parameterization of the problem. We show how a parser can be trained with a discriminative learning method while still parameterizing the problem according to a generative probability model. We present three methods for training a neural network to estimate the probabilities for a statistical parser, one generative, one discriminative, and one where the probability model is generative but the training criteria is discriminative. The latter model outperforms the previous two, achieving state-ofthe-art levels of performance (90.1 % F-measure on constituents).

Citations

3909	Neural Networks for Pattern Recognition - Bishop - 1995
1654	B.: Building a large annotated corpus of english: The Penn treebank - Marcus, Marcinkiewicz, et al. - 1993
780	Head-Driven Statistical Models for Natural Language Parsing - Collins - 1999
671	A Maximum-Entropy-Inspired Parser - Charniak - 2000
348	A maximum entropy model for part-of-speech tagging - Ratnaparkhi - 1996
254	On discriminative vs. generative classifiers: A comparison of logistic regression and Naive Bayes - Ng, Jordan - 2002
136	Learning to Parse Natural Language with Maximum Entropy Models’. Machine Learning 34:151–175 - Ratnaparkhi - 1999
45	Deterministic left corner parsing - Rosenkrantz, Lewis - 2007
43	Conditional structure versus conditional estimation in nlp models - Klein, Manning
21	2001]. “Incremental syntactic parsing of natural language corpora with simple synchrony networks - Lane, Henderson
2	Generative versus discriminative models for statistical leftcorner parsing - 2003a
2	Inducing history representations for broad coverage statistical parsing - 2003b