#### DMCA

## ReTrASE: Integrating Paradigms for Approximate Probabilistic Planning

### Cached

### Download Links

Citations: | 19 - 11 self |

### Citations

3826 | Dynamic Programming - Bellman - 1957 |

627 | Learning to act using real-time dynamic programming
- Barto, Bradtke, et al.
- 1995
(Show Context)
Citation Context ...A popular framework for formulating probabilistic planning problems is Markov Decision Processes (MDPs). One of the most popular algorithms for solving MDPs that yields high-quality solutions is RTDP =-=[1]-=-, a technique that explores the state space under the guidance of a heuristic. Unfortunately, being based on dynamic programming, it suffers from a critical drawback — it represents the value function... |

261 | Stable function approximation in dynamic programming - Gordon - 1995 |

230 | SPUDD: Stochastic planning using decision diagrams - Hoey, St-Aubin, et al. - 1999 |

170 | Efficient solution algorithms for factored MDPs - Guestrin, Koller, et al. - 2003 |

129 | Labeled RTDP: Improving the convergence of real-time dynamic programming
- Bonet, Geffner
- 2003
(Show Context)
Citation Context ...value function over the states in the trial path using Bellman backups. A popular variant, LRTDP, adds a termination condition to RTDP by labeling those states whose values have converged as ‘solved’ =-=[2]-=-. 3 ReTrASE On a high level, RETRASE explores the state space in the same manner as RTDP, but, instead of performing Bellman backups on states themselves, backups are performed over properties of the ... |

109 | Generalizing plans to new environments in relational MDPs - Guestrin, Koller, et al. - 2003 |

80 | Exponential family PCA for belief compression in POMDPs
- Roy, Gordon
- 2003
(Show Context)
Citation Context ...not clear whether it is competitive with today’s top methods. In continuous state spaces, some researchers have applied non-linear techniques like exponential-PCA and NCA for dimensionality reduction =-=[8]-=-. Most basis function based techniques are not applied in nominal domains. A notable exception is FPG [3] but RETRASE outperforms it consistently on several domains. RETRASE is described in more detai... |

79 | FF-Replan: A baseline for probabilistic planning
- Yoon, Fern, et al.
- 2007
(Show Context)
Citation Context ...sed to avoid creating a state/value table. One method involves domain determinization and uses a classical planner as a subroutine in computing a policy. Such determinization planners, e.g., FFReplan =-=[11]-=-, tend to disregard the probabilistic nature of actions and often have trouble with probabilistically interesting [7] domains. In other words, their approximation, while computationally efficient, fre... |

65 | Apricodd: Approximate policy construction using decision diagrams
- St-Aubin, Hoey, et al.
- 2001
(Show Context)
Citation Context ... Section 1) other flavors of dimensionality reduction include PCA and algebraic and binary decision diagram (ADD/BDD). In practice algorithms that use ADD/BDD do not scale to large problems. APRICODD =-=[10]-=- is an exception, but it is not clear whether it is competitive with today’s top methods. In continuous state spaces, some researchers have applied non-linear techniques like exponential-PCA and NCA f... |

60 | Symbolic heuristic search for factored Markov decision processes - Feng, Hansen - 2002 |

58 | The FF planning system: Fast plan generation through heuristic search
- Hoffman, Nebel
(Show Context)
Citation Context ...ad ends, ones for which some properties are known, and ones not yet assigned to the other two categories. When RETRASE encounters a state s of the third type, it applies a classical planner (e.g., FF =-=[5]-=-) to a determinized version of the domain starting from s. If no classical plan exists, then every probabilistic policy from s has zero probability of reaching the goal, and s is marked as a dead end.... |

47 | Exploiting first-order regression in inductive policy selection
- Gretton, Thiébaux
- 2004
(Show Context)
Citation Context ...e determinized domains for probabilistic planning, e.g. FFReplan [11] and FFHop [12]. The idea of using determinization followed by regression has parallels to some research on relational MDPs, e.g., =-=[4; 9]-=-. 7 CONCLUSION Exploiting problem structure, heuristics, and various approximations all seem to be essential components of highly scalable successful probabilistic planners. However, most existing sol... |

42 | Decision-theoretic military operations planning
- Aberdeen, Thiebaux, et al.
- 2004
(Show Context)
Citation Context ...s to goal-oriented factored MDPs 1 , defined as tuples of the form: 〈S, A, T , C, G, s0〉, where • S is a finite set of states, • A is a finite set of actions, • T is a transition function S × A × S → =-=[0, 1]-=- that gives the probability of moving from si to sj by executing a, • C is a map S × A → R + that specifies action costs, • s0 is the start state, • G is a set of (absorbing) goal states. Factored MDP... |

40 | Probabilistic planning via determinization in hindsight
- Yoon, Fern, et al.
- 2008
(Show Context)
Citation Context ...ral domains. RETRASE is described in more detail [6]. It is also related in spirit to the probabilistic planners that use determinized domains for probabilistic planning, e.g. FFReplan [11] and FFHop =-=[12]-=-. The idea of using determinization followed by regression has parallels to some research on relational MDPs, e.g., [4; 9]. 7 CONCLUSION Exploiting problem structure, heuristics, and various approxima... |

38 | mGPT: A probabilistic planner based on heuristic search - Bonet, Geffner - 2005 |

36 | Probabilistic planning vs. replanning
- Little, Thiebaux
- 2007
(Show Context)
Citation Context ...a subroutine in computing a policy. Such determinization planners, e.g., FFReplan [11], tend to disregard the probabilistic nature of actions and often have trouble with probabilistically interesting =-=[7]-=- domains. In other words, their approximation, while computationally efficient, frequently results in poor solution quality. The other method, dimensionality reduction, maps the state space to a param... |

27 | Practical linear value-approximation techniques for first-order mdps
- Sanner, Boutilier
- 2006
(Show Context)
Citation Context ...e determinized domains for probabilistic planning, e.g. FFReplan [11] and FFHop [12]. The idea of using determinization followed by regression has parallels to some research on relational MDPs, e.g., =-=[4; 9]-=-. 7 CONCLUSION Exploiting problem structure, heuristics, and various approximations all seem to be essential components of highly scalable successful probabilistic planners. However, most existing sol... |

26 | Planning with continuous resources in stochastic domains - Mausam, Brafman, et al. - 2005 |

25 | Policy generation for continuoustime stochastic domains with concurrency - Younes, Simmons - 2004 |

16 | RFF: A robust, FF-based mdp planning algorithm for generating policies with low probability of failure. 3rd International Planning Competition (IPPC-ICAPS - Teichteil-Koenigsbuch, Infantes, et al. - 2008 |

13 | The HMDPP planner for planning with probabilities. The - Keyder, Geffner - 2008 |

9 |
The factored policy gradient planner (ipc’06 version
- Buffet, Aberdeen
- 2006
(Show Context)
Citation Context ...s have applied non-linear techniques like exponential-PCA and NCA for dimensionality reduction [8]. Most basis function based techniques are not applied in nominal domains. A notable exception is FPG =-=[3]-=- but RETRASE outperforms it consistently on several domains. RETRASE is described in more detail [6]. It is also related in spirit to the probabilistic planners that use determinized domains for proba... |

8 | Preventing unrecoverable failures through precautionary planning - Foss, Onder, et al. - 2007 |

7 | and Sylvie Thiébaux. Probabilistic planning vs replanning - Little - 2007 |

3 | Shie Mannor, and Doine Precup. Automatic basis function construction for approximate dynamic programming and reinforcement learning - Keller - 2006 |

2 | Regressing deterministic plans for MDP functionapproximation. InWorkshop on A Reality Check for Planning and Scheduling Under Uncertainty at ICAPS - Kolobov, Mausam, et al. - 2008 |