W. Lehnert and B. Sundheim. A performance evaluation of text analysis technologies. AI Magazine, 12(3):81-94, 1991.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....large information sources in multiple formats. Evidence extraction (EE) is the task of obtaining structured evidence data from unstructured, natural language documents. EE builds on information extraction technology developed under DARPA s earlier MUC (Message Understanding Conference) programs [Lehnert Sundheim1991, Cowie Lehnert1996] and the current ACE (Automated Content Extraction) program at the National Institute of Standards and Technology (NIST) NIST] Link Discovery (LD) is the task of identifying known, complex, multi relational patterns that indicate potentially threatening activities in large ....

Lehnert, W., and Sundheim, B. 1991. A performance evaluation of text-analysis technologies. AI Magazine 12(3):81--94.

....large information sources in multiple formats. Evidence extraction (EE) is the task of obtaining structured evidence data from unstructured, natural language documents. EE builds on information extraction technology developed under DARPA s earlier MUC (Message Understanding Conference) programs [23, 8] and the current ACE (Automated Content Extraction) program at the National Institute of Standards and Technology (NIST) 29] Link Discovery (LD) is the task of identifying known, complex, multi relational patterns that indicate potentially threatening activities in large amounts of relational ....

W. Lehnert and B. Sundheim. A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94, 1991.

....CMU Figure 2: Query interface of BullsI Search Enter Your Search Terms: What you have entered is: Name of University Institution: A Research Interest A Teaching Interest A Person s Name 3. PRELIMINARY RESULTS Each of the information extraction tasks is evaluated using precision and recall [3]. We tagged a homepage as relevant to each of the extraction task if it contains the respective information (name, email address, research, and teaching) in textual forms. The results are shown in Table 1. Column 1 shows ten randomly selected universities that are sampled. Column 2 displays the ....

Lehnert W. & Sundheim B. "A Performance Evaluation of Text Analysis Technologies" AI Magazine pp. 81-94, 1991.

....domain can be e#ectively ignored. And since the system is only concerned with the domain specific portions of the text, some of the most di#cult problems in NLP are simplified. As a result, information extractionis a practical and feasible technology that has achieved success in the last few years [16]; 17] 18] 19] While extracting the information in our system the domain is not known, thus at first the domain would be all subjects found in LCSH, for instance. Since each subject has it own specific keywords, we try to select each and every keyword that is found in both the text and in our ....

Lehnert, W. G. and Sundheim, B. 1991. A Performance Evaluation of Text Analysis Technologies. AI Magazine 12(3):81-94. 44

....with the help of a user model and an inference mechanism, Web Mining refers to actively searching the web for what the system deduces as the preference areas of the user. A common technique used for deducing user preferences is Indexing and some of its variants like Latent Semantic Indexing or LSI [Lehnert and Sundheim, 1991]. Indexing relies on building lists of actively used keywords (or Indices) which are taken to indicate active interest areas of the user. Many systems in use currently, however, tend to rely exclusively on various types of Indexing. It was felt that a significant amount of potentially useful ....

Wendy Lehnert and B. Sundheim. A Performance Evaluation of Text-Analysis Technologies, AI Magazine, p. 81-94, 1991

....filtering of information. In many cases, manual browsing through even a limited portion of the relevant information obtainable through advancing information retrieval (IR) and information extraction (IE) technologies (Callan, Croft, Harding 1992; Larkey Croft 1996; Cowie Lehnert 1996; Lehnert Sundheim 1991) is no longer effective. The time quality cost tradeoffs offered by Copyright (c) 1998, American Association for Artificial Intelligence (www.aaai.org) All rights reserved. This material is based upon work supported by the Department of Commerce, the Library of Congress, and the National ....

Lehnert, W., and Sundheim, B. 1991. A performance evaluation of text analysis technologies. AI Magazine 12(3):81-- 94.

....shown in Figure 3. Note that the three last pattern match the same groups but with different synset numbers. This means that different semantical senses of the words match producing diffent general patterns. Recall, Precision and the mixture of them R P (also known as F with fi value set to one) [5]. In short, Recall measures the coverage of the set of IE pattern and Precision measures the quality of the IE patterns obtained. Both values are expressed as percentages. A 100 of Recall indicates that all information that had to be extracted were actually extracted. A 100 of Precision ....

Wendy Lehnert and Beth Sundheim. A performance evaluation of text analysis technologies. AI Magazine, pages 81--94, 1991.

....to automatically validate the patterns generated, releasing the expert from this task and obtaining in this way results directly comparable with other systems. Validation will be done with the known measures of Recall, Precision and the mixture of them R P (also known as F with value set to one) [5]. In short, Recall measures the coverage of the set of IE pattern and Precision measures the quality of the IE patterns obtained. Both values are expressed as percentages. A 100 of Recall indicates that all information that had to be extracted were actually extracted. A 100 of Precision ....

W. Lehnert and B. Sundheim, `A performance evaluation of text analysis technologies', AI Magazine, pp. 81--94, (1991).

....and Piti e Salpetri ere, Paris) showed a comparable result of 65 parsed sentences. Along the other dimension, we set up a method for evaluating globally the near whole set of functions of the indexing system for French. The protocol is inspired from the third Message Understanding Conference [13, 14]. The principle of this evaluation consists in comparing the behaviour of the system with the one of a human reader, disposing of the same texts, and performing the basic tasks of Menelas: information retrieval and nomenclature code generation. The first phase of the project mainly involved ....

Wendy Lehnert and Beth Sundheim. A performance evaluation of text-analysis technologies. Artificial Intelligence Magazine, (Fall):81-- 94, 1991.

....gathering planning problem that is too difficult to solve without high level filtering of information. In many cases, manual browsing through even a limited portion of the relevant information obtainable through advancing information retrieval (IR) and information extraction (IE) technologies [4, 27, 7, 28] is no longer effective. The time quality cost tradeoffs offered by the collection of information sources and the dynamic nature of the environment lead us to conclude that the user cannot (and should not) serve as the detailed controller of the information gathering (IG) process. Our solution to ....

W.G. Lehnert and B. Sundheim. A performance evaluation of text analysis technologies. AI Magazine, 12(3):81--94, 1991.

....then downloads each of these pages and uses a simple pattern based information extraction system to extract data about each title. Information extraction (IE) is the task of locating specific pieces of information from a document, thereby obtaining useful structured data from unstructured text [24, 12]. Specifically, it involves finding a set of substrings from the document, called fillers, for each of a set of specified slots. When applied to web pages instead of natural language text, such an extractor is sometimes called a wrapper [22] The current slots utilized by the recommender are: ....

W. Lehnert and B. Sundheim. A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94, 1991.

....text, the various word senses of any given word, and related semantic issues, than there is about constituent boundaries. So Semeval can be expected to turn out even roughergrained than Parseval [Bla96] According to this, the definition of what counts as a correct analyis is a hindrance (see [LS91] for an account of this problem in the realm of message understanding) Many researchers dealing with syntactic analysis have however pointed out evaluation problems are not solved when a agreement concerning a reference analysis has been found. The argument is that a criterion like the exact ....

W. Lehnert and B. Sundheim. A Performance Evaluation of Text-analysis Technologies. AI Magazine, pages 81 -- 94, 1991. muc.ps.gz.

.... organization, and location names, as well as dates, times, percentages, and monetary amounts [3] The information extraction research community has studied such tasks and reported results at annual Message Understanding Conferences (MUC) Metadata detection in scientific articles [8] or news items [9], and acronym detection [23] are among the many other problems that can very naturally be recast as tag insertion. 2 The tag insertion problem Briefly stated, the general tag insertion problem is this: 2 Given a sequence of characters C 0 . C n 1 and a set of tags T 0 . T t 1 , how ....

Wendy Lehnert. A performance evaluation of text analysis technologies. AI Magazine, pages 81--94, Fall 1991.

....to distinguish a document from its neighbors is to answer specific questions about its contents. A good number of systems have been developed that look for answers to a set of predefined questions about a specific topic domain, notably those demonstrated at the Message Understanding Conferences [Lehnert and Sundheim, 1991]. Most of these systems require large amounts of task specific domain knowledge and complex inferencing capabilities. The process of building up and representing the necessary knowledge bases is time consuming and good coverage is di#cult to achieve. For this reason, our question should be revised ....

Lehnert, W. and Sundheim, B. (1991). A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94.

....then downloads each of these pages and uses a simple pattern based information extraction system to extract data about each title. Information extraction (IE) is the task of locating specific pieces of information from a document, thereby obtaining useful structured data from unstructured text [17, 10]. Specifically, it involves finding a set of substrings from the document, called fillers, for each of a set of specified slots. When applied to web pages instead of natural language text, such an extractor is sometimes called a wrapper [15] The current slots utilized by the recommender are: ....

Wendy Lehnert and Beth Sundheim. A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94, 1991.

.... techniques that automatically acquire this lexicon from the training corpus using a symbolic induction algorithm (Thompson, 1995) We are also exploring the use of ILP methods in constructing rules for information extraction (the task of identifying specific items in a natural language text (Lehnert Sundheim, 1991)) ILP techniques can potentially induce more complex patterns than previous learning methods applied to this task (Riloff, 1996; Soderland, Fisher, Aseltine, Lehnert, 1995) By inducing unbounded relational patterns that characterize the context surrounding particular phrases, concise rules ....

Lehnert, W., & Sundheim, B. (1991). A performance evaluation of text-analysis technologies. AI Magazine, 12 (3), 81--94.

....in the correct explanation, the precision error rate P = the number of excess assumptions divided by the number of assumptions in the computed explanation, and the overall error rate O = the average of the recall and precision error rates. We used similar quality measures and terminology as in [ Lehnert and Sundheim, 1991 ] If more than one best explanations are computed for an example, we take the error rates for the example to be the average of the error rates over all the best explanations. We ran Accel on the 50 examples using two different evaluation metrics: the coherence metric (break slight ....

Wendy Lehnert and Beth Sundheim. A performance evaluation of textanalysis technologies. AI Magazine, 12(3):81--94, 1991.

....is a natural language processing task that involves automatically extracting predefined types of information from text. In contrast to in depth understanding, information extraction systems focus only on portions of text that are relevant to a specific domain (e.g. see [ Jacobs and Rau, 1990; Lehnert and Sundheim, 1991 ] For example, an information extraction system designed for a terrorism domain might extract the names of perpetrators, victims, physical targets, and weapons involved in a terrorist attack. Or an information extraction system designed for a joint ventures domain might extract the names of ....

....victims, physical targets, and weapons involved in a terrorist attack. Or an information extraction system designed for a joint ventures domain might extract the names of companies involved in a joint venture and products, facilities, or people associated with those companies. CIRCUS [ Lehnert, 1991 ] is a conceptual sentence analyzer that performs information extraction. CIRCUS uses a dictionary of concept nodes to recognize domain specific patterns and expressions and to extract relevant information. A concept node is essentially a case frame that is activated by specific linguistic ....

[Article contains additional citation context not shown here]

Lehnert, W. G. and Sundheim, B. 1991. A Performance Evaluation of Text Analysis Technologies. AI Magazine 12(3):81--94.

....objective for example, to find the best price for a music CD. Our work pushes these ideas to the next level. Our solution to the information explosion is to integrate different Artificial Intelligence (AI) technologies, namely scheduling, planning, text processing, information extraction (IE) [2, 22, 5, 23] and interpretation problem solving, into a single information gathering agent, BIG (resourceBounded Information Gathering) that can take the role of the human information gatherer. BIG locates, retrieves, and processes information to support a human decision process implementationally, BIG ....

W.G. Lehnert and B. Sundheim. A performance evaluation of text analysis technologies. AI Magazine, 12(3):81--94, 1991.

....uses the same information and hypothesis space. A classifier that performs this way is called a Bayes Optimal Classifier (Mitchell, 1997) 2. 4 Information Extraction Information Extraction attempts to apply patterns (or templates) to text in order to extract information relevant to certain areas (Lehnert Sundheim, 1991; Cardie, 1997; Califf Mooney, 1998) In essence, it attempts to take advantage of certain shallow regularities in language as a means to extract information relevant to certain highly informative fields. For instance, for text describing a conference, we may want to automatically extract ....

Lehnert, W., & Sundheim, B. (1991). A performance evaluation of text-analysis technologies. AI Magazine, 12 (3), 81--94.

....and associated background knowledge. A key feature of the formalism is that it supports the generation of multiple hypotheses and uses its knowledge sources to sift through and assess competing hypotheses. 4. 5 Further Work To learn more about information extraction techniques and systems, see [47, 36]. Several systems have been developed recently that learn dictionaries for information extraction, such as [43, 67, 74] Some older systems that incorporated symbolic learning techniques with natural language processing include [1, 29, 9, 37] Explanation based learning has also been previously ....

W. G. Lehnert and B. Sundheim. A performance evaluation of text analysis technologies. AI Magazine, 12(3):81--94, 1991.

....problem that is too difficult to solve without high level filtering of information. In many cases, manual browsing through even a limited portion of the relevant information obtainable through advancing information retrieval (IR) and information extraction (IE) technologies (Larkey Croft 1996; Lehnert Sundheim 1991) is no longer effective. The time quality cost tradeoffs offered by the collection of information sources and the dynamic nature of the environment lead us to conclude that the user cannot (and should not) serve as the detailed controller of the information gathering (IG) process. Our solution to ....

Lehnert, W., and Sundheim, B. 1991. A performance evaluation of text analysis technologies. AI Magazine 12(3):81--94.

....process such texts makes shallow text understanding methods such as information extraction (IE) the task of locating specific pieces of data from a natural language document, particularly useful. In recognition of their significance, IE systems have been the focus of DARPA s MUC program (Lehnert Sundheim 1991). Unfortunately, IE systems, although they don t attempt full text understanding, are still difficult and time consuming to build and the resulting systems generally contain highly domainspecific components, making them difficult to port to new domains. IE systems, then, are an attractive testbed ....

Lehnert, W., and Sundheim, B. 1991. A performance evaluation of text-analysis technologies. AI Magazine 12(3):81--94.

....are often stated implicitly, and even if the text explicitly mentions them the descriptions are often located far enough apart to make detection difficult. Although the importance of discourse processing for information extraction has been emphasized in the Message Understanding Conferences (Lehnert Sundheim, 1991; Hirschman, 1992) no system presented has satisfactorily addressed the issue. The discourse processor in textract is able to correlate individual pieces of information throughout the text. textract merges concepts which the pattern matcher has identified separately (and usually in different ....

Lehnert, W., & Sundheim, B. (1991). A Performance Evaluation of Text-Analysis Technologies.

....parses with sentences or semantic categories with words. The resulting corpora are much more than text: they constitute sources of knowledge about language. Typical work on annotated corpora includes: ffl evaluating NLP systems: this is how the DARPA MUC competitions have been proceeding [20]; ffl training NLP systems: e.g. a part of speech tagger can tag accurately some 97 of the words of a corpus provided it is given a large enough hand annotated training corpus [21] ffl building resources for NLP systems; this is actually a variant of the preceding point: a corpus where each ....

Lehnert W and Sundheim B. A performance evaluation of text-analysis technologies. The Artificial Intelligence Magazine 1991(Fall):81--94.

....type of knowledge, one could encode productions that are rather specific to a particular type of sentence. In fact, chunking would produce such productions that would be applicable in sentences that represent particular combinations of ambiguities and syntactic and semantic contexts. Cardie and Lehnert (1991) have extended a conceptual analyzer (such as CA described earlier) to handle complex syntactic constructs such as embedded clauses. They show that the conceptual parser can correctly interpret the complex syntactic constructs without a separate syntactic grammar or explicit parse tree ....

.... no Parallel AQUA (Ram, 1989) no yes no N A Integrated Race based parsing no no no yes Cascaded (McRoy Hirst, 1990) Competence Model (Bates no no no yes Uncontrolled MacWhinney, 1991) parallel SAL (Jurafsky, 1991) no yes no N A Integrated CIRCUS LICKS (Cardie no yes no N A Integrated Lehnert, 1991) NL SOAR (Lehman no yes in syntax yes Integrated Lewis, 1991) CC READER (Just no yes possibly N A Uncontrolled Carpenter, 1992) parallel COMPERE (Mahesh yes yes yes yes Controlled Eiselt, 1993) parallel 88 CHAPTER VI THE THEORY OF PARSING: WHEN TO COMMUNICATE WITH SEMANTICS The general ....

[Article contains additional citation context not shown here]

Lehnert, W. and Sundheim, B. (1991). A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94.

....two approaches in the sections that follow. Large scale government sponsored research on information filtering also began in this period. In 1989 the United States Defense Advanced Research Projects Agency (DARPA) sponsored the first of an ongoing series of Message Understanding Conferences (MUC) [23, 17]. 5 The principal thrust of those conferences has been use of information extraction techniques to support the selection of messages. In 1990, DARPA launched the TIPSTER project to fund the research efforts of several of the MUC participants [13] TIPSTER added an emphasis on the use of ....

Wendy Lehnert and Beth Sundheim. A performance evaluation of text analysis technologies. AI Magazine, 12(3):81--94, Fall 1991.

....of who is either the man or the woman and the man, depending on the number of the embedded clause verb. 1 Locating the implicit position, or gap, that the antecedent fills in the embedded clause is a separate, but equally difficult problem, and will not be discussed here. See Cardie and Lehnert (1991) for a solution to the gap finding problem that is consistent with the work presented here. ffl Sometimes, the antecedent is truly ambiguous. For sentences like S9, the antecedent depends on the surrounding context. ffl Locating the antecedent requires the assimilation of both syntactic and ....

....them. This has important ramifications for the case representation, which can include attribute value pairs for only those linguistic constructs and knowledge sources that are available to the NLP system. In our work, the larger NLP system is the CIRCUS information extraction system (Lehnert 1990, Lehnert et al. 1991, Lehnert et al. 1992, Lehnert et al. 1993) In general, an information extraction system takes as input a set of unrestricted texts and summarizes each text with respect to a prespecified topic or domain of interest: it finds useful information about the domain and encodes that information in a ....

[Article contains additional citation context not shown here]

W. Lehnert and B. Sundheim. 1991. A performance evaluation of text analysis technologies. AI Magazine, 12(3):81--94.

....associated costs to the available information. The complexity of the information gathering problem lies in the fact that manual navigation and browsing of even a subset of the relevant information obtainable through advancing information retrieval (IR) and information extraction (IE) technologies [2, 8, 5, 9] is ineffective without high level filtering. The time quality cost tradeoffs offered by the collection of information sources and the dynamic nature of the environment lead us to conclude that the user cannot (and should not) serve as the detailed controller of the information gathering (IG) ....

W.G. Lehnert and B. Sundheim. A performance evaluation of text analysis technologies. AI Magazine, 12(3):81--94, 1991.

....individual messages and extracts specific pieces of information for the database such as the type of job, the location, the salary, the starting date, etc. Such natural language information extraction systems have been hand built as part of ARPA s MUC (Message Understanding Conference) program (Lehnert Sundheim, 1991; ARPA, 1993) and several projects have used learning techniques to automatically acquire rules for this task (Riloff, 1993; Soderland Lehnert, 1994; Huffman, 1996) We plan to develop a system that uses ideas from ILP to learn patterns for extracting information from newsgroup postings. ....

Lehnert, W., & Sundheim, B. (1991). A performance evaluation of text-analysis technologies. AI Magazine, 12 (3), 81--94.

....in specific formats. Any unstructured database like a text database can also be converted to a case base by generating semantic descriptors characterizing each document in the database. Much of the work in information extraction and text summarization concentrates on generating such descriptors[19]. Given such descriptor generating capabilities, any set of databases with inter related data can be treated as distributed case bases. Information requirements of many real life applications lead to such distributed case bases. Let us illustrate this with another example: a venture capitalist ....

Lehnert, W. and Sundheim, B., "A Performance Evaluation of Text-Analysis Technologies," AI Magazine, 1991, pp 81-94.

.... process such texts makes information extraction (IE) the task of locating specific pieces of data from a natural language document, a particularly useful sub area of natural language processing (NLP) In recognition of their significance, IE systems have been the focus of DARPA s MUC program (Lehnert and Sundheim, 1991). Unfortunately, IE systems are difficult and time consuming to build and the resulting systems generally contain highly domain specific components, making them difficult to port to new domains. Recently, several researchers have begun to apply learning methods to the construction of IE systems ....

Lehnert, Wendy and Beth Sundheim. 1991. A performance evaluation of text-analysis technologies.

.... current natural language processing (NLP) systems cannot yet perform in depth text understanding, they can read an arbitrary text and summarize its major events provided that those events fall within a particular domain of interest (e.g. stories about natural disasters or terrorist events) [11, 17]. Thus far, among the best performing and most robust language processing systems for this type of limited summarization task have been knowledge based natural language systems NLP systems that rely heavily on domain specific, handcrafted knowledge to handle the myriad syntactic, semantic, and ....

W. Lehnert and B. Sundheim. A performance evaluation of text analysis technologies. AI Magazine, 12(3):81--94, 1991.

....feasible. 2 2 Recently researchers have become more successful at converting short, domain specific texts into template like representations. Liddy 1991) reports work on converting empirical abstracts into knowledge structures, and several of the researchers participating in the MUC competition (Lehnert Sundheim 1991) are showing promising results at classifying the contents of newswire articles. The proposal presented here is intended to allow users access to a partially structured representation of full length documents, in a framework that can be implemented automatically and relatively efficiently; these ....

Lehnert, W. & B. Sundheim (1991). A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94.

....information requires a deeper understanding of natural language. One way of providing more understanding is with information extraction. Information extraction is the task of locating specific pieces of data from a natural language document, and has been the focus of DARPA s MUC program (Lehnert Sundheim, 1991). The extracted information can then be stored in a database which could then be queried using either standard database query languages or a natural language database interface. An example of the information extraction task which was the focus of MUC 3 and MUC 4 appears in Figures 1.1 and 1.2. The ....

....the natural language processing resources used in this research. 2. 1 Information Extraction Information extraction is a shallow form of natural language understanding useful for certain types of document processing, which has been the focus of ARPA s Message Understanding Conferences (MUC) (Lehnert Sundheim, 1991; DARPA, 1992, 1993) It is useful in situations where a set of text documents exist containing information which could be more easily used by a human or computer if the information were available in a uniform database format. Thus, an information extraction system is given the set of documents ....

Lehnert, W., & Sundheim, B. (1991). A performance evaluation of text-analysis technologies. AI Magazine, 12 (3), 81--94.

....extraction (IE) Califf, 1998) The goal of an IE system is to find specific pieces of information in a natural language document. The specification of the information to be extracted generally takes the form of a template with a list of slots to be filled with substrings from the document (Lehnert Sundheim, 1991). IE is particularly useful for obtaining a structured database from unstructured documents and is being used for a growing number of web and Internet applications. Rapier is a bottom up relational learner, and acquires rules in the form of a sequence of patterns that identify relevant phrases in ....

Lehnert, W., & Sundheim, B. (1991). A performance evaluation of text-analysis technologies. AI Magazine, 12 (3), 81--94.

....BRA then downloads each of these pages and uses a simple pattern based information extraction system to extract data about each title. Information extraction (IE) is the task of locating specific pieces of information from a document, thereby obtaining useful structured data from unstructured text [16, 9]. Specifically, it involves finding a set of substrings from the document, called fillers, for each of a set of specified slots. When applied to web pages instead of natural language text, such an extractor is sometimes called a wrapper [14] The current slots utilized by the recommender are: ....

Wendy Lehnert and Beth Sundheim. A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94, 1991.

....scores were virtually indistinguishable with the AutoSlog dictionary achieving 99.7 of the performance of the handcrafted dictionary. Introduction Knowledge based natural language processing (NLP) systems have demonstrated strong performance for information extraction tasks in limited domains [Lehnert and Sundheim, 1991; MUC 4 Proceedings, 1992] But enthusiasm for their success is often tempered by real world concerns about portability and scalability. Knowledge based NLP systems depend on a domain specific dictionary that must be carefully constructed for each domain. Building this dictionary is typically a ....

Lehnert, W. G. and Sundheim, B. 1991. A Performance Evaluation of Text Analysis Technologies. AI Magazine 12(3):81--94.

....to the newsgroup misc.jobs.offered. Section 5 presents our conclusions. 2 Background 2. 1 Information Extraction Information extraction is the task of locating specific pieces of data from a natural language document, and has been the focus of ARPA s Message Understanding Conferences (MUC) Lehnert and Sundheim, 1991; ARPA, 1992; 1993 ] Usually the data to be extracted is described by a template specifying a list of slots to be filled. For example, Figure 1 shows part of a job posting, and the corresponding slots of the filled computer science job template. IE can be useful in a variety of domains. The ....

Wendy Lehnert and Beth Sundheim. A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94, 1991.

....of the IE process, but can also be applied to text mining. The MUC conferences make extensive use of these measurements in judging entries. Overgeneration and Fallout The MUC conferences also measured overgeneration, which is the degree of superfluous information extracted by the systems [Leh91] Again we have the problem of deciding what is superfluous, and what is critical a human decision. The MUC test data is designed to get around this problem by chosing a fairly restricted problem domain (Government reports on Latin American terrorism) in which superfluous information should be ....

Wendy Lehnert. A performance evaluation of text analysis technologies. AI Magazine, pages 81--94, Fall 1991.

....at this time. A serious description of the MUC 3 tests and evaluation becomes fairly involved and cannot be covered in the space of a few paragraphs. For our purposes we ll show some sample input, some sample output, and a sample score report. A more comprehensive overview can be found in [Lehnert and Sundheim 1991]. Here is a sample text taken from the first blind test set (TST1) TST1 MUC3 0004 BOGOTA, 30 AUG 89 (INRAVISION TELEVISION CADENA 2) TEXT] LAST NIGHT S TERRORIST TARGET WAS THE ANTIOQUIA LIQUEUR PLANT. FOUR POWERFUL ROCKETS WERE GOING TO EXPLODE VERY CLOSE TO THE TANKS WHERE 300,000 GALLONS ....

....Analyzer CIRCUS is a conceptual analyzer that produces semantic case frame representations for input sentences. Although space does not permit us to give a full technical description of CIRCUS, we will attempt to convey some sense of sentence analysis via CIRCUS. For more details, please consult [Lehnert 1991; and Cardie and Lehnert 1991] CIRCUS uses no syntactic grammar and produces no parse tree as it analyzes a sentence. Rather, it uses lexically indexed syntactic knowledge to segment incoming text into noun phrases, prepositional phrases, and verb phrases. These constituents are stored in global ....

[Article contains additional citation context not shown here]

Lehnert, W.G. and Sundheim, B. (1991) "A Performance Evaluation of Text Analysis Technologies," AI Magazine. Fall 1991. pp. 81-94.

....and Lessons (1991) A serious description of the MUC 3 tests and evaluation becomes fairly involved and cannot be covered in the space of a few paragraphs. For our purposes we ll show some sample input, some sample output, and a sample score report. A more comprehensive overview can be found in [Lehnert and Sundheim 1991]. Here is a sample text taken from the TST1 test set: TST1 MUC3 0004 BOGOTA, 30 AUG 89 (INRAVISION TELEVISION CADENA 2) TEXT] LAST NIGHT S TERRORIST TARGET WAS THE ANTIOQUIA LIQUEUR PLANT. FOUR POWERFUL ROCKETS WERE GOING TO EXPLODE VERY CLOSE TO THE TANKS WHERE 300,000 GALLONS OF THE ....

....had succeeded in producing a system that showed tremendous promise. Here is an excerpt from their MUC 4 site report: The inspiration for FASTUS was threefold. First, we were struck by the strong performance that the group at the University of Massachusetts got out of a fairly simple system [Lehnert et al. 1991]. It was clear they were not doing anything like the depth of preprocessing, syntactic analysis, or pragmatics that was being done by the systems at SRI, General Electric, or New York University. They were not doing a lot of processing. They were doing the right processing. p. 268, Sundheim 92] ....

[Article contains additional citation context not shown here]

Lehnert, W.G. and B. Sundheim. (1991). "A Performance Evaluation of Text Analysis Technologies", AI Magazine, Fall 1991. pp. 81-94.

No context found.

W. Lehnert and B. Sundheim. A performance evaluation of text analysis technologies. AI Magazine, 12(3):81-94, 1991.

No context found.

W. Lehnert and B. Sundheim. A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94, 1991.

No context found.

Lehnert, W., and Sundheim, B. 1991. A performance evaluation of text-analysis technologies. AI Magazine 12(3):81--94.

No context found.

W. Lehnert and B. Sundheim. A performance evaluation of text-analysis technologies. AI Magazine, 12(3):81--94, 1991.

No context found.

W.G. Lehnert, B. Sundheim, A performance evaluation of text analysis technologies, AI Magazine 12 (3) (1991) 81--94.

No context found.

Lehnert W and Sundheim B. A performance evaluation of text-analysis technologies. The Artif Intell Mag 1991; (Fall):81--94.

No context found.

Wendy Lehnert and B. Sundheim, A Performance Evaluation of TextAnalysis Technologies," AI Magazine , p. 81-94., 1991

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC