DMCA
Conservation decision-making in large state spaces
| Citations: | 1 - 0 self |
BibTeX
@MISC{Chadès_conservationdecision-making,
author = {I. Chadès and S. Linke and H. P. Possingham},
title = {Conservation decision-making in large state spaces},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract: For metapopulation management problems with small state spaces, it is typically possible to model the problem as a Markov decision process (MDP), and find an optimal control policy using stochastic dynamic programming (SDP). SDP is an iterative procedure that seeks to optimise a value function at each timestep by trying each of the actions defined in the MDP. Although SDP gives the optimal solution to conservation management questions in a stochastic world, its applicability has always been limited by the so-called curse of dimensionality. The curse of dimensionality is the problem that adding new state variables inevitably results in much larger (often exponential) increases in the size of the state space, which can make solving superficially small problems impossible. A large state space makes optimal SDP solutions computationally expensive to compute because optimal SDP techniques require the value function to be updated for the entire state space for every time step. The high computational requirements of large SDP problems means that only simple population management problems can be analysed. In this paper we present an application of the on-line sparse sampling algorithm proposed by Kearns, Mansour & Ng (2002), which can be used to approximate the optimal solution of a MDP for a given starting state. The algorithm is particularly attractive for problems with large state spaces as it has a running time that is independent of the size of the state space of the problem.
Keyphrases
large state space conservation decision-making optimal solution value function state space so-called curse high computational requirement mansour ng optimal control policy stochastic world small state space new state variable on-line sparse running time simple population management problem time step metapopulation management problem management question optimal sdp solution markov decision process entire state space stochastic dynamic programming optimal sdp technique iterative procedure large sdp problem small problem
