Borrajo, D., & Veloso, M. (1997). Lazy incremental learning of control knowledge for e#ciently obtaining quality plans. AI Review, 11, 371--405.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....and mutation operators. L2Plan was tested on two domains, the blocks world domain and the briefcase domain. In both domains, L2Plan was able to produce control knowledge that allowed the planner to find solutions to all of the test problems. 2 Previous Work There have been many methods [1, 4, 5, 10] used for learning control knowledge. The two most relevant works are Khardon s L2Act system [6, 7] and Aler et al. s EvoCK system [1, 2] Khardon s system, L2Act [6, 7] represented the control knowledge as an ordered list of existentially quantified rules [7] known as a Production Rule ....

....is a sophisticated domain independent search based planner. One of its features is that it allows the user to supply domain specific control knowledge to be used to guide its decision making process. It is this control knowledge that EvoCK generates. EvoCK uses heuristics generated by HAMLET [4] for Prodigy4.0 and evolves them to produce better heuristics. These heuristics are converted by EvoCK into control rules which are then used to generate the initial population for EvoCK s GP. The candidates are sets of different control rules. The EvoCK GP then uses various mutation and crossover ....

Borrajo, D., and Veloso, M. (1997) Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review 11(1-5), 371--405.

....the mainstream work in reinforcement learning by the use of a relational representation. Relational representations are commonly used in planning approaches. There have also been some attempts to combine planning with relational learning within those approaches, e.g. within the PRODIGY approach [2]. Our approach is related to them through the use of a relational representation. However, it seems that the combination of planning, reinforcement learning and relational learning has not been addressed so far. The reinforcement learning part of the work presented in this paper is admittedly ....

Borrajo, D., and Veloso, M. (1997) Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review, 11(1-5): 371--405.

....traces. Similarly to EBL search control rules, our learned plan rewriting rules also su#er from the utility problem #Minton 1988#. PbR addresses both planning e#ciency and plan quality. Some systems also learn search control that addresses both these concerns #Estlin Mooney 1997; Borrajo Veloso 1997; P#erez 1996#. However, from the reported experimental results on that work, PbR seems to be signi#cantly more scalable. 0.01 0.1 1 10 100 1000 10000 0 5 10 15 20 25 30 35 40 45 50 Average Planning Time (CPU Seconds) Number of Packages PbR Manual PbR Learned Initial IPP #a# ....

Borrajo, D., and Veloso, M. 1997. Lazy incremental learning of control knowledge for e#ciently obtaining quality plans. AI Review 11:371#405.

....traces. Similarly to EBL search control rules, our learned plan rewriting rules also suffer from the utility problem (Minton 1988) PbR addresses both planning efficiency and plan quality. Some systems also learn search control that addresses both these concerns (Estlin Mooney 1997; Borrajo Veloso 1997; P erez 1996) However, from the reported experimental results on that work, PbR seems to be significantly more scalable. 0.01 0.1 1 10 100 1000 10000 0 5 10 15 20 25 30 35 40 45 50 Average Planning Time (CPU Seconds) Number of Packages PbR Manual PbR Learned Initial IPP (a) ....

Borrajo, D., and Veloso, M. 1997. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review 11:371--405.

.... have long studied the problem of automatically creating e#cient planners by learning domain specific rules or cases to control a general search engine (Minton, 1988; Carbonell, Knoblock, Minton, 1990; Veloso, 1992; Etzioni, 1993; Bhatnagar Mostow, 1994; Kambhampati, Katukam, Qu, 1996; Borrajo Veloso, 1997; Aler, Borrajo, Isasi, 1998; Leckie Zukerman, 1998; etc. However, the successful practical application of machine learning techniques has been limited by at least two factors: First, traditional domain independent planning systems (e.g. PRODIGY, SOAR, NONLIN, UCPOP) scale so poorly that ....

Borrajo, D. & Veloso, M. M. (1997) Lazy incremental learning of control knowledge for e#ciently obtaining quality plans. In David Aha (Ed.), Lazy Learning , Kluwer Academic Publishers.

.... are promising showing that more complex problems are solvable and that planning can be speed up considerably when applying macros (Precup Sutton 1998) In classical planning, learning is currently investigated mainly in the context of the acquisition of domain specific control knowledge (Borrajo Veloso 1996). We consider recursive macros as a special case of control knowledge. Classical linear macros restrict search by offering sequences of operators which can be applied instead of primitive actions. Iterative or recursive macros (Shell Carbonell 1989; Shavlik 1990) ideally eliminate search ....

.... l) PRE f(at o l) at Rocket l)g ADD f(inside o Rocket)g DEL f(at o l)g move rocket PRE f(at Rocket A)g ADD f(at Rocket B)g DEL f(at Rocket A)g unload(o l) PRE f(inside o Rocket) at Rocket l)g ADD f(at o l)g DEL f(inside o Rocket)g 1987) and conditional planning (Peot Smith 1992; Borrajo Veloso 1996): instead of a plan representing a sequence of actions transforming a single initial state into a state fulfilling the top level goals, DPlan constructs a planning tree, representing optimal action sequences for all states belonging to the planning domain. A planning tree represents the same ....

Borrajo, D., and Veloso, M. 1996. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal 10:1--34.

....or when new methods are introduced into the domain, no re implementation of the affected methods is necessary. Moreover, the declarative representation of control information by rules can be a basis for automatically learning control rules, as realized in some planning systems, e.g. in [31, 1, 13]. Control rules are new in proof planning and in automated theorem proving. In problem solving planning, however, SOAR [12] and then Prodigy [32] were the first systems which used control rules. My experience and discussions at CMU and Edinburgh were very helpful in order to find and bring about ....

D. Borrajo and M. Veloso. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 10:1--34, 1996.

....to their case retrieval and revision needs. For example, CAPlan CbC (Mu noz Avila Hullen, 1996) learns feature weights to index plan descriptions, and RUNNER (Seifert et al. 1994) learns operator application conditions and uses them to index cases represented by semantic networks. Hamlet (Borrajo Veloso, 1997) learns to improve its search efficiency and resulting plan quality by incrementally refining its control rules using a CBL approach. Meta AQUA (Cox Ram, 1994) uses explicit learning goals to select strategies for recovering from planning failures. Krovvidy and Wee s (1993) CBL system saves ....

....to ensure that the retrieved case solutions are appropriate. 3. 3 Components in integrated learning frameworks CBL algorithms have been integrated with learning algorithms that target a variety of performance tasks, including knowledge acquisition (Tecuci, 1993) analytic problem solving (Borrajo Veloso, 1997), reinforcement learning (McCallum, 1995) and Bayesian reasoning (Tirri et al. 1996) These integrations frequently use CBL in innovative fashions. For example, McCallum s cases provide historical information that allow agents to distinguish perceptually identical locations, while Borrajo and ....

Borrajo, D., & Veloso, M. (1997). Lazy incremental learning of control knowledge for efficiently obtaining quality plans. To appear in Artificial Intelligence Review.

....(1995) explained how storing case his8 tories of exploration traces can help to solve state aliasing problems. Analytic learning: Some research on explanation based learning has examined the utility of lazy approaches for deduction (e.g. Tadepalli (1989) Clark Holte (1992) More recently, Borrajo and Veloso (1997) have shown that a lazy learning approach can be used to efficiently acquire control knowledge in an incremental planning process. 2.1.4 Summary This section briefly described investigations of research related to CBR in a few disciplines, but neglected research in many others. For example, ....

Borrajo, D., & Veloso, M. (1997). Lazy incremental learning of control knowledge for efficiently obtaining quality plans. Artificial Intelligence Review, 11, 371--405.

....adds further complications for learning search control not addressed by these early systems, since many more search paths must be considered in generating an explanation. Unfortunately, very few learning systems have been built to acquire control knowledge for other styles of planning. Hamlet (Borrajo Veloso, 1997) is one more recent system, which learns control knowledge for the nonlinear planner Prodigy4.0 (Carbonell et al. 1992) Similar to Scope, Hamlet uses a combination of EBL with induction to acquire control rules. First, optimal solutions are gathered for each training example by performing an ....

....the efficiency performance of the Prodigy4.0 planner in the blocksworld and logistics planning domains, however, since Hamlet and Scope have been tested on completely different planning algorithms, only a very rough comparison can be drawn. When examining the results reported for Hamlet in Borrajo and Veloso (1997) and the results reported for Scope reported in Chapter 6 of this dissertation, the following was noted. In blocksworld, Scope achieved a speedup factor of 23x on problems containing 4 6 goals, while Hamlet achieved a speedup of 2x on problems containing 5 goals and a speedup of 35x on more ....

Borrajo, D., & Veloso, M. (1997). Lazy incremental learning of control knowledge for efficiently obtaining quality plans. Artificial Intelligence Review, 11, 371--405.

No context found.

Borrajo, D. and Veloso, M.: 1997, Lazy incremental learning of control knowledge for efficiently obtaining quality plans, AI Rev. J. Special Issue on Lazy Learning 11(1--5), 371--405.

No context found.

Borrajo D. and Veloso M. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, Vol. 11 pages 371-405, 1997. 13

No context found.

Daniel Borrajo and Manuela Veloso. Lazy incremental learning of control knowledge for eciently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 11(1-5):371-405, February 1997.

....very little has been done in learning for problem solving on showing how this a ects the results. We used a subset of prodigy4.0 control knowledge language, the so called select rules, in earlier experiments. The reason was that evock was used in combination with another learning system (hamlet [ Borrajo and Veloso, 1997 ] that was able to use only select rules. In this paper, we present empirical results in the blocksworld domain where evock is used to explore a larger and exible control knowledge language subset, including prefer and reject rules. We also describe for the rst time the grammar related aspects ....

....(arm empty) add (holding ob ) add (clear underob ) can keep incorrect individuals for longer, and they stand a chance of being corrected. This result could only be obtained by exploiting grammar evock richness and declarativeness. Other control knowledge learning methods, such as hamlet [ Borrajo and Veloso, 1997 ] use ad hoc grammars in the sense that they have them programmed. Therefore, exploring this type of hypothesis requires a heavy programming e ort. For instance, incorporating new types of rules or metapredicates implies having to re program their code. 5 Examples of individuals The aim of ....

Daniel Borrajo and Manuela Veloso. Lazy incremental learning of control knowledge for eciently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 11(1-5):371{ 405, February 1997.

....domain is to learn search control knowledge to guide the planning process. There have been different approaches to acquiring control knowledge for non trivial (nonlinear) planning. Some of them use analogy [7, 17] others deduction [8, 12] induction [10] and some combine deduction and induction [2, 5]. Fully deductive methods have their weaknesses. For instance, they usually require a complete domain theory. Alternatively, inductive approaches incrementally acquire correct knowledge by observing a large set of Copyright c fl1998, American Association for Artificial Intelligence ....

....by observing a large set of Copyright c fl1998, American Association for Artificial Intelligence (www.aaai.org) All rights reserved. In fact, planning is intractable [4] problem solving examples. These approaches have strong bias built in their search operators. For instance, Hamlet [2] learned control knowledge has the property that, among many different possible generalizations, the most specific generalization is always selected. A learning paradigm with lighter bias might be desirable. Genetic Programming (GP) 9] is such a paradigm. This article presents a GP based ....

[Article contains additional citation context not shown here]

Daniel Borrajo and Manuela Veloso. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 11(1-5):371--405, February 1997.

....learning those heuristics by experience on solving previous problems. For this purpose, one could try to use any of the previous work on search control knowledge. Some of them use analogy [10,19] others pure deduction [11,13] pure induction [12] and some combine deduction and induction [5,7]. However, while planning to obtain a solution plan has been largely studied in the literature, the search for optimal or good plans has been usually discarded due to its complexity. During the last decade, there has been some work that applies learning techniques for improving not only the ....

.... not only the eciency of the planning task on nding solutions, but also the quality of those solutions, according to some prede ned criteria [9,15,17] hamlet is one such systems that learns control knowledge to guide eciently a non linear planner, in our case prodigy4.0 [18] to good solutions [5]. We showed how it was possible to improve both criteria (eciency and quality of plans) However, plan quality was measured in terms of length of the solutions. This is not enough in many real world problems in which people is interested on obtaining a reduction in terms of costs, time, resources ....

[Article contains additional citation context not shown here]

Daniel Borrajo and Manuela Veloso. Lazy incremental learning of control knowledge for eciently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 11(1-5):371-405, February 1997.

....In particular, we propose a twostage loosely coupled multi strategy system (Hamlet EvoCK) that learns control knowledge for Prodigy4.0, a means ends bidirectional planner. The rst stage Hamlet is an incremental deductive inductive multi strategy system itself, built by Borrajo and Veloso [5]. The second stage (EvoCK: Evolution of Control Knowledge) is a GP based system whose purpose is to add corrective biases to Hamlet. Hamlet EvoCK is loosely coupled in the sense that the output of the rst stage is just fed into the second stage, by seeding the GP initial population, instead of ....

....in terms of complementary biases. Subsection 3.3 describes how they are actually combined. Finally, Subsections 3.4, 3.5, and 3.6 explain in detail EvoCK learning biases. 3. 1 Hamlet Hamlet is an incremental learning method based on EBL [25] Explanation Based Learning) and inductive re nement [5]. Figure 2 shows Hamlet modules and their connection to Prodigy4.0. P p in P C L L,D C C D Bounded Explanation module CONTROL Refinement module PRODIGY ST C ST HAMLET ST ST Fig. 2. Hamlet high level architecture. Inputs to Hamlet are: the domain description D, the set of ....

[Article contains additional citation context not shown here]

Daniel Borrajo and Manuela Veloso. Lazy incremental learning of control knowledge for eciently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 11(1-5):371-405, February 1997.

....They can modify the system behaviour if the obtained solutions are successful to the user queries. These learning modules are di erent to UserAgents learning modules, in a PlannerAgent learning could be used to gain eciency in planning processes, to obtain better solutions learning control rules [2], or retrieving and adapting old stored solutions in the own agent to avoid the planning process itself [15] Planning module: Works to solve the user problem. Currently the planning module uses a non lineal planner named prodigy4.0[14] The PlannerAgents use a planner as the main reasoning ....

D. Borrajo and M. Veloso. Lazy incremental learning of control knowledge for eciently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 11(1-5):371-405, February 1997.

.... used in linear problem solving (independence of goals) they are hard to generalize in the case of nonlinear problem solving, and, in particular, planning, where it is dicult to capture correct explanations of the interactions among goals, multiple planning operator choices, and situational data [2]. With respect to the learning task, we preferred exploring the learning of control knowledge instead of domain knowledge. In this paper, we use hamlet, a relational system that learns control knowledge for a non linear planner. 1 hamlet uses a lazy multistrategy approach, combination of ....

....task, we preferred exploring the learning of control knowledge instead of domain knowledge. In this paper, we use hamlet, a relational system that learns control knowledge for a non linear planner. 1 hamlet uses a lazy multistrategy approach, combination of analytical and inductive techniques [2]. As most relational learning systems, it learns by generalizing and specializing a set of rules clauses using an ad hoc de nition of subsumption. The terms ILP and relational learning have sometimes been used indistinctly. We will use ILP as a subclass of systems that perform relational ....

Daniel Borrajo and Manuela Veloso. Lazy incremental learning of control knowledge for eciently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 11(1-5):371-405, February 1997.

....for learning to act in dynamic,uncertain worlds, and we briefly discuss it in the next section. Machine learning techniques have been studied extensively to improve the efficiency of classical AI planning systems that do not include any representation of uncertainty (Minton 1988; Veloso 1994; Borrajo Veloso 1996; Estlin Mooney 1996; Katukam Kambhampati 1994) and these techniques are still useful for reducing search in domains with uncertainty. Some of the machine learning methods used in classical planners find new uses for the conditional plans now considered. Analogy, for example, can be used very ....

....slow as tank barge1. This example illustrates the learning opportunities available in planning under uncertainty. In general, our learning method allows Weaver to learn from experience which features of the problem to pay attention to in choosing the steps in a plan. The method is based on HAMLET (Borrajo Veloso 1996) which is a learning algorithm that combines deductive and inductive techniques to acquire control rules to improve the planning efficiency and the quality of the plans generated. 1 In this section, we briefly introduce HAMLET s learning technique and then present how we apply and HAMLET as an ....

[Article contains additional citation context not shown here]

Borrajo, D., and Veloso, M. 1996. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. Artificial Intelligence Review in press.

....We implemented most of the plans transformations described above. The implementation did not requires changes to the Prodigy architecture. This is because Prodigy allows control of the planning search to be manipulated through declarative structures, called control rules (Veloso et al. 1995; Borrajo Veloso 1996). Control rules enable heuristic redirection of the search for operators to achieve goals, for bindings to instantiate operators, and for the next node to be expanded. The default is to perform depthlimited search through planning decisions. But given a control rule that selects a specific search ....

Borrajo, D., and Veloso, M. M. 1996. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning 10:1--34.

....function. We propose the use of background knowledge coming from the use of a previous learning technique also in another two search elements [ Aler et al. 1998a ] first, the initial state will not be created randomly, but using control knowledge learned by another method, hamlet in this case [ Borrajo and Veloso, 1997 ] Second, genetic operators will use knowledge in the form of examples, obtained as a sub product of hamlet learning process. In [ Aler et al. 1998a ] we have shown that GP obtains much better results in planning by using such background knowledge. The purpose of this paper is to show that a ....

....complex problems need to be given a high node limit if they are to be solved. As many such evaluations must be performed for each generation, only simple problems can be used for learning (otherwise the fitness function would take too long) This is another bias to take into account. 3 However, Borrajo and Veloso, 1997 ] shows empirically that training with simple problems is enough for learning control knowledge useful to solve more complex problems. 4 EXPERIMENTAL RESULTS In order to test our multi strategy approach, the following steps were carried out: 1. Hamlet was trained with 400 learning planning ....

[Article contains additional citation context not shown here]

Daniel Borrajo and Manuela Veloso. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 11(1-5):371--405, February 1997.

No context found.

Borrajo, D., & Veloso, M. (1997). Lazy incremental learning of control knowledge for e#ciently obtaining quality plans. AI Review, 11, 371--405.

No context found.

Borrajo, D., and Veloso, M. 1997. Lazy incremental learning of control knowledge for e#ciently obtaining quality plans. AI Review 11:371--405.

No context found.

Daniel Borrajo and Manuela M. Veloso. Lazy incremental learning of control knowledge for e#ciently obtaining quality plans. Artificial Intelligence Review, 10:1--34, 1996.

No context found.

Borrajo, D., Veloso, M.: Lazy Incremental Learning of Control Knowledge for E#- ciently Obtaining Quality Plans. Artificial Intelligence Review 10 (1996) 1--34

No context found.

Springer. Borrajo, D. and Veloso, M. (1997). Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review, 11:371--405.

No context found.

Daniel Borrajo and Manuela Veloso. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 10:1--34, 1996.

No context found.

Daniel Borrajo and Manuela Veloso. Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning, 10:1--34, 1996.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC