Birnbaum, L., Collins, G., Freed, M., &Krulwich, B. #1990#. Model-based diagnosis of planning failures. In Proceedings of the Eighth National ConferenceonArti#cial Intelligence,pp.318#323 Boston, MA. AAAI.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....general approach uses a model of the system s own reasoning processes to detect and to explain reasoning failures #see also Fox Leake #1995c##. Using model based reasoning to represent the reasoning task itself was proposed by Birnbaum et al. and implemented by them for self debugging planners #Birnbaum, Collins, Freed, Krulwich, 1990#. They suggested that model based introspective reasoning could be applied to case based reasoning systems #Birnbaum, Collins, Brand, Freed, Krulwich, Pryor, 1991#. However, their ideas were never implemented for casebased reasoning, and their approach has never been empirically evaluated. This ....

....highly abstract expectations. The justi#cation structures form the link between Background 38 abstract expectations and actual performance. The kind of model based reasoning Birnbaum et al. propose was applied previously to self debugging planners which used rule chaining to construct plans #Birnbaum et al. 1990#. Their approach was also implemented in CASTLE, a system which introspectively learned strategies for playing chess #Collins et al. 1993; Krulwich et al. 1992#. A variation was applied to Firby s reactive planning domain #RAPTER, #Freed Collins, 1994a#, see below#. Their proposal for ....

[Article contains additional citation context not shown here]

Birnbaum, L., Collins, G., Freed, M., &Krulwich, B. #1990#. Model-based diagnosis of planning failures. In Proceedings of the Eighth National ConferenceonArti#cial Intelligence,pp.318#323 Boston, MA. AAAI.

....and Woetzel 1987; Auramaki and Leppanen 1989; Karbe and Ramsberger 1990; Strong 1992; Mi and Scacchi 1993; Ellis, Keddara et al. 1995; Klein 1998) manufacturing control (Parthasarathy 1989; Katz 1993; Visser 1995) model based fault diagnosis (deKleer and Williams 1986; Krishnamurthi and Jr. 1989; Birnbaum, Collins et al. 1990; Friedrich, Gottlob et al. 1990) planning (Sussman 1973; Goldstein 1975; Broverman and Croft 1987; Birnbaum, Collins et al. 1990) and failure mode analysis research (Raheja 1990) Most workflow research has focused on languages for expressing correctness preserving transforms on workflow ....

....Klein 1998) manufacturing control (Parthasarathy 1989; Katz 1993; Visser 1995) model based fault diagnosis (deKleer and Williams 1986; Krishnamurthi and Jr. 1989; Birnbaum, Collins et al. 1990; Friedrich, Gottlob et al. 1990) planning (Sussman 1973; Goldstein 1975; Broverman and Croft 1987; Birnbaum, Collins et al. 1990), and failure mode analysis research (Raheja 1990) Most workflow research has focused on languages for expressing correctness preserving transforms on workflow models, providing no guidance however concerning which transforms to use for a given situation. There has been some manufacturing and ....

Birnbaum, L., G. Collins, et al. (1990). Model-Based Diagnosis of Planning Failures. Proceedings of the National Conference on Artificial Intelligence (AAAI-90).

....design incorporates only the most basic capabilities. A few efforts have done some preliminary exploration of the use of distinct exception handling services. This work has occurred predominantly in the context of business process enactment [1 3, 5, 6] manufacturing control [7 9] and planning [10, 11]. The process enactment and manufacturing work, in general, has either not evolved to the point of constituting a computational model, or has been applied to a very limited range of domains (e.g. just software engineering or flexible manufacturing cell control) and exception types (e.g. just ....

Birnbaum, L., et al. Model-Based Diagnosis of Planning Failures. in AAAI-90. 1990.

....component. Related work can also be found if we go farther afield into such disciplines as planning, distributed systems, 15 manufacturing process control, and the like. There has also been substantial work in the planning and robotics communities on dealing with unexpected world states [35] 36] [37] [38] 39] 40] This work focuses almost exclusively on exceptions (e.g. failed operations, unexpected events) in the world manipulated by the agents, and not on exceptions concerning the agents themselves. Finally, there has been substantial work on detecting and resolving exceptions in ....

L. Birnbaum, G. Collins, M. Freed, and B. Krulwich, "Model-Based Diagnosis of Planning Failures," presented at Proceedings of the National Conference on Artificial Intelligence (AAAI-90), 1990. 20

.... E Mentally Enables Mentally Results Bark domain co domain Object Bark domain co domain Object animate domain co domain Decision Basis Figure 6: XP Novel Situation Alternative Refuted 14 8 Related work Several past studies have called for meta level reasoning (e.g. Collins Birnbaum, 1990; Davis, 1980; Stefik, 1981; Wilensky, 1981) and several conferences have been held on the subject (e.g. Brazdil Konolige, 1990; Maes Nardi, 1988) However, traditional approaches to meta reasoning stress knowledge about a system s own knowledge and reasoning in the form of rules dealing ....

....the learning process, and, furthermore, that they should be addressed in a unified learning framework based on introspection and Meta XPs. The approach is to use an analysis of reasoning failures encountered to determine what needs to be learned. In this respect, the approach is similar to that of Birnbaum, Collins, Freed Krulwich (1990), Mooney Ourston s (1991) EITHER system, and Park Wilkins s (1990) ODYSSEUS and MINERVA systems, but with some important differences. ODYSSEUS only deals with what are called novel situations in Meta AQUA; when it fails to explain an action, it always assumes that relevant facts or rules are ....

[Article contains additional citation context not shown here]

Birnbaum, L., Collins, G., Freed, M., and Krulwich, B. "Model-based diagnosis of planning failures," Proceedings of the Eighth National Conference on Artificial Intelligence, pp. 318--323, Boston, MA, 1990.

....cost of the efficiencies that can result from coordinationmechanism specific, albeit domain independent, exception handling mechanisms. There has also been substantial work in the planning and robotics communities on dealing with unexpected world states (Traverso, Spalazzi et al. 1996) Howe 1995) (Birnbaum, Collins et al. 1990) (Broverman and Croft 1987) Firby 1987) Hindriks, de Boer et al. 1998) This work focuses almost exclusively on exceptions (e.g. failed operations, unexpected events) in the world manipulated by the agents, and not on exceptions concerning the agents themselves. Finally, there has been ....

Birnbaum, L., G. Collins, et al. (1990). Model-Based Diagnosis of Planning Failures. Proceedings of the National Conference on Artificial Intelligence (AAAI-90).

....the time consuming and pain staking task of building the individual base cases. 7 Related Work There is a large project underway in the Institute of Learning Sciences at Northwestern University named Castle. This project takes a Model Based Reasoning (MBR) approach to computer chess ( 4] 5] [1]) The ILS approach is a very knowledge intensive one, and involves planning, learning and high level reasoning. Kerner has recently published work on a CBR approach to chess ( 8] However, his work focuses on developing a case base evaluation function for a standard searching chess system, and ....

Lawrence Birnbaum, Gregg Collins, Michael Freed, and Bruce Krulwich. Modelbased diagnosis of planning failures. In Proceedings of the Eighth National Conference on Artificial Intelligence, pages 318--323. American Association of Artificial Intelligence, AAAI Press/MIT Press, 1990.

.... cooperative work (Mi and Scacchi 1993) Chiu, Karlapalem et al. 1997) Klein 1998) Auramaki and Leppanen 1989) Finkelstein, Gabbay et al. 1994) manufacturing control (Fletcher and Misbah 1999) Adamides and Bonvin 1993) Katz 1993) planning (Traverso, Spalazzi et al. 1996) Howe 1995) (Birnbaum, Collins et al. 1990) (Broverman and Croft 1987) Firby 1987) and multi agent systems (Hgg 1996) Kaminka and Tambe 1997) Venkatraman and Singh 1999) Bansal, Ramohanarao et al. 1997) The result is that good ideas developed within one discipline do not readily propagate to researchers and practitioners in other ....

Birnbaum, L., G. Collins, et al. (1990). Model-Based Diagnosis of Planning Failures. Proceedings of the National Conference on Artificial Intelligence (AAAI-90).

....may not only be less efficient, but it may be worse than no learning at all. Finally, Section 6 presents related research and summarizes our contributions. 2. Representation and theory Failure provides both human and artificial reasoners with strong clues when deciding what needs to be learned [1,20,30,35,37,41,44,67,87,90,93,95]. One of the major goals of establishing a theory of introspective learning, therefore, is to provide both a general characterization of reasoning failure and the potential causes of such failure in order to discover the nature of these clues. A sufficient characterization of failure will ....

....as determined from the output of the performance task and contained in the reasoning trace. That is, blame assignment is like troubleshooting; it is a mapping function from failure symptom to failure cause. The purpose is the same whether the troubleshooter is explaining a broken device or itself [1,93]. The input trace describes how results or conclusions were produced by specifying the prior causal chain (both of mental and physical states and events) The learner retrieves an abstract metaexplanation pattern, or IMXP, from memory and applies it to the trace in order to produce a specific ....

[Article contains additional citation context not shown here]

L. Birnbaum, G. Collins, M. Freed and B. Krulwich, Model-based diagnosis of planning failures, in: Proceedings of the Eighth National Conference on Artificial Intelligence (AAAI Press, Menlo Park, CA, 1990) 318-323.

....embodied in the Process Handbook. This work also constitutes, we believe, a substantive and novel contribution to previous efforts on exception handling, which have been pursued in the context of workflow [1, 9, 13, 18, 22, 25] manufacturing control [14, 23, 26] model based fault diagnosis [3, 7, 19], planning [3, 4] and failure mode analysis research [24] Most workflow research has focused on languages for expressing correctness preserving transforms on workflow models, providing no guidance however concerning which transforms to use for a given situation. There has been some manufacturing ....

....Process Handbook. This work also constitutes, we believe, a substantive and novel contribution to previous efforts on exception handling, which have been pursued in the context of workflow [1, 9, 13, 18, 22, 25] manufacturing control [14, 23, 26] model based fault diagnosis [3, 7, 19] planning [3, 4], and failure mode analysis research [24] Most workflow research has focused on languages for expressing correctness preserving transforms on workflow models, providing no guidance however concerning which transforms to use for a given situation. There has been some manufacturing and workflow ....

L. Birnbaum, G. Collins, M. Freed and B. Krulwich. Model-Based Diagnosis of Planning Failures. Proceedings of the 8 th National Conf. on Artificial Intelligence (AAAI-90), pp.31823, 1990.

....hypotheses are generated, the strategies chosen for verifying candidate hypotheses, and the basis for choosing particular reasoning methods for each of these. Trace Meta XPs are similar to reasoning traces [Carbonell, 1986; Minton, 1988; Veloso and Carbonell, 1991] or justification structures [Birnbaum, Collins, Freed, and Krulwich, 1990; deKleer, Doyle, Steele, and Sussman, 1977; Doyle, 1979] with the difference that Trace Meta XPs represent, in addition to the subgoal structure of the problem and justifications for operator selection decisions, information about the structure of the (possibly multistrategy) reasoning process ....

....learn. One form of introspection that has been implemented in many systems is the use of reasoning traces to represent problem solving performance; an early example of this approach was Sussman s [1975] HACKER program. Reasoning trace information has primarily been used for blame assignment (e.g. [Birnbaum, Collins, Freed, and Krulwich, 1990]) and for speedup learning (e.g. Mitchell, Keller, and Kedar Cabelli, 1986] In addition, we propose that such information, suitably augmented with the kinds of knowledge represented in our Introspective Meta XP structures, can be used as the basis for the selection of learning strategies in a ....

[Article contains additional citation context not shown here]

L. Birnbaum, G. Collins, M. Freed, and B. Krulwich. Model-Based Diagnosis of Planning Failures. In Proceedings of the Eighth National Conference on Artificial Intelligence, pages 318--323, Boston, MA, 1990.

....in response to the system s failing to handle some new example. Unfortunately, new knowledge must be entered in a form that is very close to fully operational. Moreover, failure diagnosis typically requires the user to have a detailed knowledge of reasoning process of the underlying system. See [ Birnbaum et al. 1990 ] and [ Smith et al. 1985 ] however, for attempts to automate diagnosis completely. Existing systems employ model based reasoning to help users manage the complexity of entering data at the operational level, as opposed to facilitating an informal, high level dialog that sidesteps this ....

Birnbaum, Lawrence; Collins, G.; Freed, M.; and Krulwich, B. 1990. Modelbased diagnosis of planning failures. In Proceedings AAAI-88 Eighth National Conference on Artificial Intelligence, Boston. AAAI. 318--323.

.... to the comparison of design and planning case based systems, case based diagnosis system can rely on planning and thus can be classified as a planning system (Turner, 1989) However, there are also diagnosis systems which are model based (Koton, 1988b; Goel, 1989; Goel and Chandrasekaran, 1989; Birnbaum et al. 1990; F eret and Glasgow, 1993) In these cases, categorization is easier since the distinctive features are easily perceivable. A brief description of some case based diagnosis systems follows: ffl SHRINK (Kolodner and Kolodner, 1987) is an early case based reasoning system, designed to be a ....

.... Collins, 1989; Hunter, 1989; Owens, 1989; Hammond, 1989a) or is based on the representation hierarchy (Porter et al. 1988; Lauzon and Rose, 1993) ffl knowledge can be stored as is or a generalization hierarchy may be created (Shinn, 1988; Branting, 1989; Thompson and Langley, 1989; Collins and Birnbaum, 1990; Fertig and Gelertner, 1991; Jones, 1991; Yang, Robertson and Lee, 1993; Ram, 1993b) ffl the algorithm can depend heavily on a special computer architecture (Stanfill and Waltz, 1986; Kolodner, 1989; Waltz, 1989; Rubin, 1992) or it can be implemented on a standard computer; ffl the ....

[Article contains additional citation context not shown here]

Birnbaum, L., Collins, G. C., Freed, M., and Krulwich, B. (1990). Model-based diagnosis of planning failures. In Proc. of AAAI-90, pages 318--323, Boston, MA.

.... has occurred predominantly in the context of business process enactment (Kreifelts and Woetzel 1987; Auramaki and Leppanen 1989; Karbe and Ramsberger 1990; Strong 1992; Mi and Scacchi 1993) manufacturing control (Parthasarathy 1989; Katz 1993; Visser 1995) and planning (Broverman and Croft 1987; Birnbaum, Collins et al. 1990). The process enactment and manufacturing work, in general, has either not evolved to the point of constituting a computational model, or has been applied to a very limited range of domains (e.g. just software engineering or flexible manufacturing cell control) and exception types (e.g. just ....

Birnbaum, L., G. Collins, et al. (1990). Model-Based Diagnosis of Planning Failures. AAAI.

.... an observed anomaly (e.g. Leake, 1992; Ram, 1991) While such goals are often implicitly programmed into machine learning systems by the system s designers, or provided as input by the system s users, it is also possible for a machine learning system to determine its own learning goals (e.g. Birnbaum, Collins, Freed Krulwich, 1990; desJardins, 1992; Hunter, 1989; Ram, 1989, 1991; Ram Cox, 1994; Redmond, 1992) For example, if a system engaged in some real world problem solving task encounters difficulties during the performance of the task, it can reason about the knowledge it brought to bear, and the reasoning processes ....

Birnbaum, L., Collins, G., Freed, M., & Krulwich, B. (1990). Model-Based Diagnosis of Planning Failures. In Proceedings of the Eighth National Conference on Artificial Intelligence, pages 318--323, Boston, MA.

.... understanding (DeJong, 1986; Leake, 1992; Mooney, 1990; Ram, 1993; Schank, 1986; Wilensky, 1983) in planning and problem solving, explanations are used to identify the sources of problems and successes, to repair current problems, exploit successful solutions, and avoid problems in the future (Birnbaum et al. 1990; Hammond, 1989; Leake, 1992; Minton, 1988) in multistrategy learning, explanations are used to guide selection of learning methods (Hunter, 1990b; Hunter, 1990a; Ram and Cox, 1994) In addition, explanations are also often used by learning systems to guide selection of appropriate indices for ....

.... to be explained is provided as an external input to the system (e.g. Clancey and Shortliffe, 1984; de Kleer and Williams, 1989; Ng and Mooney, 1990) Other systems generate their own explanation tasks in response to processing failures, expectation failures, or performance failures (e.g. (Birnbaum et al. 1990; Hammond, 1989; Leake, 1992; Ram, 1993; Ram and Cox, 1994; Riesbeck, 1981; Schank, 1986) However, information needs may arise even in the absence of failures per se. For example, the policy to attempt to improve performance at a task may motivate learning even if current performance matches ....

Birnbaum, L., Collins, G., Freed, M., and Krulwich, B. (1990). Model-based diagnosis of planning failures. In Proceedings of the Eighth National Conference on Artificial Intelligence, pages 318--323, Boston, MA. AAAI.

....behavior characterized by the model. The approach used here was proposed by Birnbaum et al. Birnbaum, Collins, Brand, Freed, Krulwich, Pryor, 1991) who have applied modelbased methods to the generation of self debugging planners in other contexts (Collins, Birnbaum, Krulwich, Freed, 1993; Birnbaum, Collins, Freed, Krulwich, 1990). A number of issues arise in applying this approach, including how to represent the model, the levels of abstraction at which the model should characterize system behavior, and how to apply the model to detecting and repairing problems. In this paper, we focus primarily on one aspect of the ....

....failures of the device. That model can be expressed in the form of assertions that are true when the model is functioning correctly. Although model based reasoning is usually used to diagnose and repair the failures of a device to perform as expected, it has been observed (Collins et al. 1993; Birnbaum et al. 1990) that model based reasoning can also be used to characterize ideal system behavior beyond expectations for actual performance in order to identify deficiencies and guide learning to bring the system closer to the ideal. Birnbaum et al. sketch this approach with a hypothetical example involving ....

Birnbaum, L., Collins, G., Freed, M., & Krulwich, B. (1990). Modelbased diagnosis of planning failures. In Proceedings of the Eighth National Conference on Artificial Intelligence, pp.

.... in an explanation, as well as constraints associated with specific types of problems that can arise in explanations with different types of content) The effort to formulate this taxonomy can build on current research on characterizing reasoning failures (e.g. Leake, 1992; Kass, 1990; Birnbaum et al. 1990; Ram et al. 1992 ] 2. Building a model of planful memory search. Previous research has developed methods for flexible memory traversal through an index elaboration process [ Kolodner, 1984 ] We propose going further in treating the search for adaptation knowledge as a planning process in ....

....adaptation strategies from scratch. However, because it controls memory search with an explicit planning process that is accessible to system reasoning, the framework makes memory search strategies themselves accessible to failuredriven plan repair methods (e.g. Sussman, 1975; Hammond, 1989; Birnbaum et al. 1990 ] Although existing models of failure driven plan repair should be readily applicable to this task, one new issue that arises is how to detect the memory search problems themselves. The issue here is one of identifying and diagnosing failures of the system s own internal processing (e.g. ....

[Article contains additional citation context not shown here]

Birnbaum, L.; Collins, G.; Freed, M.; and Krulwich, B. 1990. Model-based diagnosis of planning failures. In Proceedings of the Eighth National Conference on Artificial Intelligence, Boston, MA. AAAI. 318--323.

....for the two combination Meta XPs in the table, E 0 represents the concept that should have been predicted but was not, and M 0 represents the memory item that should have been retrieved but was not. 8 Related work Several past studies have called for meta level reasoning (e.g. Collins Birnbaum, 1990; Davis, 1980; Stefik, 1981; Wilensky, 1981) and several conferences have been held on the subject (e.g. Brazdil Konolige, 1990; Maes Nardi, 1988) However, traditional approaches to meta reasoning stress knowledge about a system s own knowledge and reasoning in the form of rules dealing ....

....the learning process and, furthermore, that they should be addressed in a unified learning framework based on introspection and Meta XPs. The approach is to use an analysis of reasoning failures encountered to determine what needs to be learned. In this respect, the approach is similar to that of Birnbaum, Collins, Freed Krulwich (1990), Mooney Ourston s (1993) EITHER system, and Park Wilkins s (1990) ODYSSEUS and MINERVA systems, but with some important differences. ODYSSEUS only deals with what are called novel situations in Meta AQUA; when it fails to explain an action, it always assumes that relevant facts or rules are ....

[Article contains additional citation context not shown here]

Birnbaum, L., Collins, G., Freed, M., & Krulwich, B. "Model-based Diagnosis of Planning Failures," Proceedings of the Eighth National Conference on Artificial Intelligence, pp. 318--323, Boston, MA, 1990.

....specific definition can now be provided. 4.2.1 Operationalized Phases of Learning As the introduction argues in Section 1, there are three fundamental learning problems, all of which must be addressed for effective learning in open world scenarios. They are the credit or blame assignment problem (Birnbaum, Collins, Freed, Krulwich, 1990; Freed, Krulwich, Birnbaum Collins, 1992; Minsky, 1963; Stroulia, Shankar, Goel Penberthy, 1992; Weintraub, 1991) deciding what to learn (Hunter, 1989, 1990; Keller, 1986; Krulwich, 1991; Leake Ram, to appear; Ram, 1991; Ram Hunter, 1992) and the strategy selection problem (Cox Ram, ....

....to the rest of the story) 62 5. 2 Artificial Intelligence Related Research Our approach to using an analysis of reasoning failures to determine what needs to be learned is similar to Mooney and Ourston s (1991) EITHER system, Park and Wilkins (1990) MINERVA program, the CASTLE system of Birnbaum et al. 1990), and Stroulia and Goel s (1992) METAROUTER, but with some important differences. We have focused on the use of meta models for explicit representation of domain knowledge and of reasoning processes in an integrated, multistrategy reasoning and learning system. Unlike Mooney and Ourston and Park ....

[Article contains additional citation context not shown here]

Birnbaum, L., and Collins, G., Freed, M. and Krulwich, B. (1990). Model-Based Diagnosis of Planning Failures. In Proceedings of the Eighth National Conference on Artificial Intelligence, Boston, MA, pp. 318-323.

....thereby identify what aspects of the system need to be altered in order to achieve the expected performance. We have previously applied this paradigm to the development of self debugging planners in competitive domains (see, e.g. Birnbaum and Collins, 1988; Collins, Birnbaum, and Krulwich, 1989; Birnbaum, Collins, Freed, and Krulwich, 1990). In this paper, we show how the paradigm can be applied to case based planning in particular. 2 Modelling case based planning In order to build a case based planner capable of using a model based approach to improving its performance, we must first develop an explicit model of case based ....

Birnbaum, L., Collins, G., Freed, M., and Krulwich, B. 1990. Model-based diagnosis of planning failures. Proceedings of the 1990 AAAI Conference, Boston, MA, pp. 318-323.

....of flaws they detect and the strategies they apply to repair the faults. Related efforts, all of which rely on causal models of the planner, include Hudlicka and Lesser s work on diagnosing failures during execution [14] and Birnbaum et al. s proposal to enhance model based diagnosis of plans [3]. Most of the research addresses debugging plans, rather than debugging the planner. While the two are related (after all, explaining why a plan would fail is a step toward explaining why a planner should not favor such a plan) little has been done on planner debugging. The most notable exception ....

Lawrence Birnbaum, Gregg Collins, Michael Freed, and Bruce Krulwich. Model-based diagnosis of planning failures. In Proceedings of the Eight National Conference on Artificial Intelligence, pages 318--323, Boston, MA, July 1990.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC