J. Lester and B. Porter. 1997. Developing and empirically evaluating robust explanation generators: The KNIGHT experiments. Computational Linguistics, 23(1):65--101.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....more specific rule to augment. the action of its parent in the hierarchy, yielding much the same behavior as with systemic network traversal; nevertheless, it should be emphasized that this is not required in a classification based approach. Turning now to schema based approaches such as that of [Lester Porter 97] beyond the obvious differences of representation and the absence of classification, one way in which our approach differs is that we have explicitly embraced a powerful object oriented programming language, rather than simply embedding a handful of procedural constructs. Since schemas are ....

Lester, J. C. and B. W. Porter. 19971 Developing and Empirically Evaluating Robust Explanation Generators: The KNIGHT Experiments.. Co,zputatioal Linguistics, vol. 23, no. 1, pages 65-100.

....not just apply rules but be able to retrieve and manipulate them as objects in their own right. For example, the system should be able to describe, as well as apply, what it knows. This is essential for many tasks, such as: description generation, for example as performed by the Knight system [9]) Knight would generate answers to questions such as Tell me about cactus plants . natural language processing (NLP) NLP systems often need to know what sort of things can fill di#erent roles, in order to disambiguate sentences (e.g. the agent in an eating event is an animate object, the ....

J. C. Lester and B. W. Porter. Developing and empirically evaluating robust explanation generators: The knight experiments. Computational Linguistics, 1996. (to appear). http://www.cs.utexas.edu/users/mfkb/papers/coling.ps.Z.

....new research questions. In the field of natural language generation, empirical evaluation has only recently become a top research priority (Dale, Eugenio et al. 1998) Some empirical work has been done to evaluate models for generating descriptions of objects and processes from a knowledge base (Lester and Porter March 1997), text summaries of quantitative data (Robin and McKeown 1996) descriptions of plans (Young to appear) and concise causal arguments (McConachy, Korb et al. 1998) However, little attention has been paid to the evaluation of systems generating evaluative arguments, communicative acts that attempt ....

Lester, J. C. and B. W. Porter (March 1997). "Developing and Empirically Evaluating Robust Explanation Generators: The KNIGHT Experiments." Computational Linguistics 23(1): 65-101.

....LGM is a more complicated matter. As Dale Mellish (1998) have pointed out, evaluation of natural language generation systems is still in its infancy, and there are no wellestablished evaluation methods in this area. An evaluation method which seems promising is the one adopted by Coch (1996) and Lester Porter (1997). They compared computer generated texts to texts from human authors by having a panel of judges, who did not know the source of the texts, rate their quality on several dimensions. However, see Dale Mellish (1998) for a discussion of some problems related to such a black box evaluation. In ....

Lester, J., & Porter, B. 1997. Developing and empirically evaluating robust explanation generators: the KNIGHT experiments. Computational Linguistics, 23(1), 65--102.

....of the system, namely reasoning (as opposed to explanation) Presumably, it is difficult for one and the same person to be a domain expert and a expert on communication in the domain. 2 We do not consider explanation generation from data bases (for example, McKeown, 1985; Paris, 1988; Lester and Porter, 1997)) to be the same problem as expert system reasoning explanation (even though we may use some similar techniques) In data base explanations, the knowledge is static and its representation is given a priori as part of the problem statement. In expert system explanations, the knowledge to be ....

....domain knowledge would never include plan operators related to the hearer s cognitive state because the hearer is not part of the domain. CDK is not a new concept. Many researchers have identified the need for packaging domain knowledge differently for communication. For example, the views of Lester and Porter (1997) can be seen as a form of CDK, though they are not a declarative representation. What is new in our work, however, is the proposal that CDK should be represented explicitly in a distinct representation from the domain knowledge. At CoGenTex, we have used an Intermediate Knowledge Representation ....

Lester, J. C. and Porter, B. W. (1997). Developing and empirically evaluating robust explanation generators: The knight experiments. Computational Linguistics, 23(1):65--102.

....where the results of processing can be expressed as annotations on text spans. With NLG, there is no such agreed basis. The only way to be sure of having realistic input is probably to take input that is provided by another system developed for purposes unrelated to NLG. This was achieved by Lester and Porter (1997) by making use of an independently developed knowledge base in the domain of biology; in certain domains (such as those the reporting of stock market behaviour or weather forecasting) numerical input may be available from independent sources. But experience shows that an application that wants to ....

Lester, J. C. and Porter, B. W. (1997) "Developing and Empirically Evaluating Robust Explanation Generators: The KNIGHT Experiments", Computational Linguistics Vol 23, No 1.

....in presenting content and how those intentions can be achieved. As it commonly happens, there is a trade off between the two approaches. Schemata are less powerful, but are easier to write than plan operators, and planners using schematas are generally more efficient than plan based text planners [LP95] On the other hand, the latter are more principled, but they are still at the prototype stage. Moreover, as [YM94] points out, plan based text generators have rarely if ever been formally assessed in terms of soundness and completeness, and the basis for writing plan operators has often been ....

....orders. Recall that, as discussed earlier, the process of text generation is divided into text planning, sentence planning, and surface realization. 6.1 Text planning for Technical Orders In Sec. 5.1. 1, we discussed three different approaches to text planning: those based on schemata [McK85, LP95] those based on planning by means of plan operators [Hov88, MP93, WAB 91] and those that don t plan for the global structure of the text, but generate text incrementally by means of local strategies [Sib92] We already mentioned in Sec. 5.1.1 that a local approach such as Sibun s doesn t ....

James C. Lester and Bruce W. Porter. Developing and empirically evaluating robust explanation generators: the KNIGHT experiments, 1995. Journal submission.

....level of detail: Similarly, the level of detail can be dynamically controlled to suit the user s level of expertise, by controlling how much information from the knowledge base is included in the answer. Although this was not explored in this project, this has been illustrated elsewhere (e.g. [1]) Adaptability to users preferred learning style: By integrating instructional material, documentation on the science underlying the experiment, and knowledgebased question answering, the user has multiple ways of exploring the material to suit his her own learning style. During this project, ....

James C. Lester and Bruce W. Porter. Developing and empirically evaluating robust explanation generators: The knight experiments. Computational Linguistics, 23(1):65--101, 1997.

....a large corpus can be extremely time consuming. Furthermore, in many situations such a corpus may not exist or may be difficult to obtain. Human judges An alternative method to evaluate generation models is to have judges score outputs of those models. This method has been proposed in [21] to evaluate NLG models. To compare two models, each judge in a panel is given outputs generated by both models 14 and is then asked to rate the outputs on several dimensions of text quality. For instance, in [21] texts were judged with respect to: overall quality and coherence, content, ....

....is to have judges score outputs of those models. This method has been proposed in [21] to evaluate NLG models. To compare two models, each judge in a panel is given outputs generated by both models 14 and is then asked to rate the outputs on several dimensions of text quality. For instance, in [21] texts were judged with respect to: overall quality and coherence, content, organization, writing style and correctness. Obviously, to guarantee integrity the judges must be unaware of which text is generated by which model. Having a panel of judges combats (but does not eliminate) the inherent ....

[Article contains additional citation context not shown here]

James C. Lester and Bruce W. Porter. Developing and empirically evaluating robust explanation generators: The knight experiments. Computational Linguistics, 23(1):65--101, March 1997.

....context of ongoing research on knowledge base construction by composition. Elsewhere we have discussed: motivations for the approach and algorithms [5, 6, 7] a graphical user interface [8] a knowledge representation and reasoning system [6] question answering and explanation generation [17, 24] Within that context, this paper provides a brief tour of an early version of our component library to highlight its requirements, construction, contents and applications. In the following section, we will describe our research project in more detail and the design constraints it places on our ....

Lester, J. and Porter, B. Developing and Empirically Evaluating Robust Explanation Generators: The KNIGHT Experiments. Computational Linguistics 23, 1 (1997), 65-101.

....of ongoing research on knowledge base construction by composition. Elsewhere we have discussed: motivations for the approach and algorithms [5, 6, 8] a graphical user interface [8] a knowledge representation and reasoning system [6] question answering methods and explanation generation [18, 25] Within that context, this paper provides a brief tour of an early version of our component library to highlight its requirements, contents, and applications. In the following section, we will describe our research project in more detail and the design constraints it places on our component ....

Lester, J. and Porter, B. Developing and Empirically Evaluating Robust Explanation Generators: The KNIGHT Experiments. Computational Linguistics 23, 1 (1997), 65-101.

....was encoded by a botany expert. His goal was to encode fundamental textbook knowledge that can support a wide range of tasks, not just prediction. In fact, the same knowledge base has been used successfully for other tasks, such as answering description questions and generating English text [31, 33, 32]. Second, the domain knowledge he encoded is extensive: it describes 700 properties of a prototypical plant and 1500 influences among them, including many different levels of detail. Finally, the questions used to evaluate tripel were produced by the botany expert, who judged tripel s models by ....

....of botany knowledge. Finally, it was developed to support a wide range of tasks besides prediction; that is, the bkb encodes fundamental, textbook knowledge, and the representation of that knowledge was not chosen to facilitate its use for any single task such as prediction. Lester and Porter [31, 33, 32] describe results on using the bkb to answer other types of questions. Using the bkb, the domain expert constructed a system description for a prototypical plant and its environment (i.e. surrounding soil and atmosphere) Most elements of the description were generated via automated inference ....

James Lester and Bruce Porter. Developing and empirically evaluating robust explanation generators: The knight experiments. Computational Linguistics. Forthcoming.

....associated with it, which can be submitted to the inference engine to find that information in the context of the current scenario. Figure 6 shows some examples of answer schema used. This approach is similar to (and inspired by) the use of Explanation Design Packages in the Knight system [8], and also similar in style to the use of schemata in other question answering systems (e.g. 9] 7] 4.2.2 Inference When a schema is to be filled in, the queries it contains are sent to the inference engine, and the answers computed and returned to the schema. To answer these queries, the ....

James C. Lester and Bruce W. Porter. Developing and empirically evaluating robust explanation generators: The knight experiments. Computational Linguistics, 23(1):65--101, 1997.

....when the knowledge base was constructed. Earlier research on one largescale project the Botany knowledge base project (Porter et al. 1988) shows that if detailed, declarative representations of concepts are available, then sophisticated question answering performance can be achieved (Lester Porter 1997; Rickel Porter 1997) However, manually constructing such representations is laborious, and this proved to be a major bottleneck in the project; moreover, it is simply not possible to anticipate all the concept representations that may be needed for answering questions. This points to a ....

Lester, J. C., and Porter, B. W. 1997. Developing and empirically evaluating robust explanation generators: The knight experiments. Computational Linguistics 22(3).

....Reeves and Nass, 1992] Animated pedagogical agents [Rickel and Johnson, 1997a, Stone and Lester, 1996] constitute an important category of animated agents whose intended use is educational applications. A recent large scale empirical study suggests that these agents can be pedagogically effective [Lester et al. 1997b] Moreover, it was determined that students perceived the agent as being very helpful, credible, and entertaining [Lester et al. 1997a] A key problem posed by lifelike agents that inhabit artificial worlds is deictic believability. In the same manner that humans refer to objects in their ....

....of animated agents whose intended use is educational applications. A recent large scale empirical study suggests that these agents can be pedagogically effective [Lester et al. 1997b] Moreover, it was determined that students perceived the agent as being very helpful, credible, and entertaining [Lester et al. 1997a] A key problem posed by lifelike agents that inhabit artificial worlds is deictic believability. In the same manner that humans refer to objects in their environment through combinations of speech, locomotion, and gesture, animated agents should be able to move through their environment, point ....

[Article contains additional citation context not shown here]

Lester, J. C. and Porter, B. W. (1997). Developing and empirically evaluating robust explanation generators: The KNIGHT experiments. Computational Linguistics, 23(1):65--101.

....dismissal without prejudice, because the judgment may become final at some later time. 4 Related Work The explanation community has extensively studied the process of planning and realizing text given a set of discourse specifications. Over the past decade, their research on discourse planning [McK85, Par88, Hov93, Caw92, Moo95, LP97] has produced a variety of techniques for determining the content and organization of many genres of text. Perhaps because of the necessity of coping with the myriad underlying rhetorical, illocutionary, and argument structures in discourse generation, this work has yielded a variety of mechanisms ....

James C. Lester and Bruce W. Porter. Developing and empirically evaluating robust explanation generators: The KNIGHT experiments. Computational Linguistics, 23(1):65--101, 1997.

.... Beginning with work on schemata [McKeown, 1982, Paris, 1988] the field has matured over the past decade and a half to produce top down discourse planners [Moore and Swartout, 1991, Suthers, 1991, Cawsey, 1992, Maybury, 1992, Hovy, 1993, Moore and Paris, 1993] and hybrid models [Suthers, 1991, Lester and Porter, 1997]. We discuss each of these in turn. The schema based approach to discourse generation began with the pioneering dissertation of McKeown [McKeown, 1982] in which she analyzed naturally occurring texts to develop a set of schemata for describing concepts. Schemata are ATN like structures that ....

....he demonstrates how the complexities of explanation planning can be dealt with in a coherent framework. Lester and Porter developed the hybrid approach of explanation design packages (EDPs) for Knight, a robust discourse generator for large scale knowledge bases [Lester and Porter, 1996, Lester and Porter, 1997]. Knight s EDPs, which constitute a schema like programming language for discourse knowledge engineers, combine a hierarchical frame based representation with embedded procedural constructs for knowledge base access. 4.2 Representing Document Planning Knowledge Document planners can build on ....

Lester, J. C. and Porter, B. W. (1997). Developing and empirically evaluating robust explanation generators: The KNIGHT experiments. Computational Linguistics, 23(1):65-- 101.

No context found.

J. Lester and B. Porter. 1997. Developing and empirically evaluating robust explanation generators: The KNIGHT experiments. Computational Linguistics, 23(1):65--101.

No context found.

James C. Lester and Bruce W. Porter. 1997. Developing and empirically evaluating robust explanation generators: The knight experiments. Computational Linguistics, 23(1):65--102.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC