Abstract

Introduction The experiment described here was designed for two things: to test the feasibility of a learning approach to information extraction in a real-world domain, and to uncover evidence that by using multiple learners it is possible to achieve better performance than by using a single learner. Because the documents used in this experiment are taken unmodified from a real online environment designed for human-to-human communication, the task is a challenging one. Its difficulty varies considerably from field to field, but in all cases, in order to conclude that this approach is feasible, I require of each learner that its performance is substantially better than that of a random guesser. Of course, in practice the required performance level is defined by the intended application. Consequently, my argument for feasibility is informal. Some applications may be able to exploit a well-behaved precision-recall curve, so I look for this from the learners tested here. We cannot

Citations

1216	Term-weighting approaches in automatic text retrieval - Salton, Buckley - 1988
784	Learning logical definitions from relations - Quinlan - 1989
728	Relevance feedback in information retrieval - Rocchio - 1971
682	The CN2 induction algorithm - Clark, Niblett - 1989
666	An introduction to hidden Markov models - Rabiner, Juang - 1986
517	Generalization as search - Mitchell - 1982
365	A sequential algorithm for training text classifiers - Lewis, Gale - 1994
351	Syntactic Structures - Chomsky - 1957
313	Parsing english with a link grammar - Sleator, Temperley - 1993
291	Rule induction with cn2: Some recent improvements - Clark, Boswell - 1991
207	Automated learning of decision rules for text categorization - Apte, Damerau, et al. - 1994
186	An evaluation of phrasal and clustered representations on a text categorization task - Lewis - 1992
136	Crystal: Inducing a conceptual dictionary - Soderland, Fisher, et al. - 1995
102	Information extraction as a basis for high-precision text classification - Riloff, Lehnert - 1994
86	Best-First model merging for hidden Markov model induction - Stolcke, SM - 1994
61	Learning to classify English text with ILP methods - Cohen - 1996
56	The generic information extraction system - Hobbs - 1992
49	Document and passage retrieval based on hidden markov models - Mittendorf, Schäuble - 1994
37	Learning decision lists using homogeneous rules - Segal, Etzioni - 1994
29	Wrap-up: a trainable discourse module for information extraction - Soderland, Lehnert - 1994
23	Automatically Acquiring Conceptual Patterns without an Automated Corpus - Riloff, Shoen - 1995
21	UMASS/HUGHES: Description of the CIRCUS system used for MUC-5 - Lehnert, McCarthy, et al. - 1993
15	Forming grammars for structured documents: An application of grammatical inference - Ahonen, Mannila, et al. - 1994
12	Text filtering in MUC-3 and MUC-4 - Lewis, Tong - 1992
12	Using White Space for Automated Document Structuring - Rus, Summers - 1995
11	Description of the JV-FASTUS System as Used for MUC-5 - APPELT, HOBBS, et al. - 1993
11	Finite-state approximations of grammars - Pereira - 1990
11	Grammatical inference: An introductory survey - Vidal - 1994
11	Comparing Human and Machine Performance for Natural Language Information Extraction: Results for English Microelectronics from the MUC-5 Evaluation - Will - 1993
10	SRI international: description of the FASTUS system used for MUC4 - Hobbs, Tyson, et al. - 1992
8	GE-CMU: Description of the Shogun System Used for MUC-5 - Jacobs, Krupka, et al. - 1993
6	BBN: Description of the PLUM System as Used for MUC-5 - Weischedel - 1993
4	GE NLToolset: Description of the system as used for MUC-4 - Krupka, Jacobs, et al. - 1992
3	CRL/BRANDEIS: Description of the Diderot System as Used for MUC-5 - Cowie, Guthrie - 1993
2	Hughes Research Laboratories: description of the trainable text skimmer used for MUC-4 - August, Dolan - 1992
2	NEC: Description of the VENIEX System as Used for MUC-5 - Muraki, Doi - 1993
1	TRW: description of the DEFT system as used for MUC-5 - Noah, Weeks - 1993