Hanson, F.B., Techniques in Computational Stochastic Dynamic Programming, in Leondes, C.T., editor, Control and Dynamic Systems Volume 76: Stochastic Digital Control System Techniques, Academic Press, New York, pp. 103-162, 1996.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....values. A wider range of stochastic control applications can be accurately modeled since the arrival rates and amplitudes can depend on the value of the state. The state dependent vector valued marked Poisson noises are related to the Poisson random measure (see Gihman and Skorohod [4] or Hanson [6]) and are defined as O 4 ; DUc O c 6c 4 # 1 j h 9 which consists of independent differentials of space time Poisson processes that are functions of the state, 4 Uc is the Poisson jump amplitude random variable or the mark of the O ....

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103-162, 1996.

....discrete time analysis while looking at an application involving groundwater remediation. The groundwater remediation problem itself has been presented as either a continuous time or discrete time problem; both have been examined analytically and numerically from the control standpoint such as in [10] and [4] This formu lation utilizes a Gaussian noise process to represent uncertainties in the level of contamination and allows for new discrete random sources for contamination to oc cur via a compound Poisson process. The focus in this paper is the remediation of the aquifer subject to a ....

F. B. Hanson, Techniques in Computational Stochastic Dynamic Programming in Control and Dynamic Systems, vol. 76, 1996, pp 103-162.

....at specified nodes. The implicit equations governing the regular or optimal controls in (25) and the scheduled jump controls in (38) require some iteration procedure such as Newton s method or can be be solved within the general extrapolatorpredictor corrector procedure summarized by Hanson [6] for computational stochastic dynamic programming problems for Markov noise in continuous time. Also, the separation imperfection function U in (27) in general requires interpolation to evaluate the value function when #L = U is not an U node. Also, ....

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103-162, 1996.

....of complex random phenomena. The inclusion of state dependence in the Poisson noise means that the quasi LQGP dynamics (1) is nonlinear in the state vector. The state dependent vector valued marked Poisson noises are related to the Poisson random measure (see Gihman and Skorohod [8] or Hanson [10]) and are defined as 3 O Y , 4 , 562; Q562 8 3 Y i , 4 , 562P 6562Enq o L F i Y i , 3 i 4 , 562; 3 562F q do L (2) 3 for Za t to which consists of XPY independent differentials of space time Poisson processes that are functions of the state, 4 , 562 , where i is ....

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103-162, 1996.

....that lead to large fluctuations in risk sensitive market assets. The space time differential Poisson processes : are related to Poisson random measure or 2 space time Poisson processes, L S , see Ito [9] Gihman and Skorohod [3] Snyder and Miller [16] or Hanson [5]; see Hanson and Tuckwell [8] for some biological examples) 0 60160 6 S ] S ) 3) for N B gM( D D D , where ; 6 is the th Poisson jump amplitude function corresponding to the th stock price, S h S 6 ( gi S ....

....specified nodes. 10 The implicit equations governing the regular or optimal controls in (22) and the scheduled jump controls in (34 35) require some iteration procedure such as Newton s method or can be be solved within the general extrapolator predictor corrector procedure summarized by Hanson [5] for computational stochastic dynamic programming problems for Markov noise in continuous time. 7. Conclusions In this paper, the portfolio optimization model for investment wealth dependent on external jump events introduced by Rishel [15] has been improved and generalized. The underlying stock ....

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103162, 1996.

....and amplitudes may depend of the state of the system. Additionally, this formulation allows for simpler dynamical system modeling of complex random phenomena. The state dependent vector valued marked Poisson noises are related to the Poisson random measure (see Gihman and Skorohod [5] or Hanson [7]) and are defined as 1 000 p E4qT9 000 [ rsY ) t u v r w xzy E9q T y 24 520 0 [ r Y ) 6) for H to I which consists of DFE independent differentials of space time Poisson processes that are functions of the state, where y is the ....

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103-162, 1996.

.... 13, 14] the following state independent vector valued marked Poisson noise is used, dP(t) dP i (t) q Theta1 ; which consists of q independent differentials of space time Poisson processes that are related to the Poisson random measure, P i (dz i ; dt) see Gihman and Skorohod [3] or Hanson [4]; see Hanson and Tuckwell [7] for some biological examples) dP i (t) Z Z i z i P i (dz i ; dt) 1) where z i is the Poisson jump amplitude random variable or the mark of the dP i (t) Poisson process where i = 1 to q, with mean or expectation: Mean[dP(t) Zdt; 2) where (t) is the ....

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103-162, 1996.

....in the Poisson noise means that the problem is not strictly a LQGP problem in the dynamics and so it is assumed that this state dependence is not dominant. The state dependent vector valued marked Poisson noises are related to the Poisson random measure (see Gihman and Skorohod [9] or Hanson [11]) and are defined as F RU ) y R d ( I ) 9 kEHjA 4 E RG d ( kE j A (2) for V to which consists of QSR independent differentials of space time Poisson processes that are functions of the state, where is the Poisson ....

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103-162, 1996.

....amplitudes may depend of the state of the 3 system. Additionally, this formulation allows for simpler dynamical system modeling of complex random phenomena. The state dependent vector valued marked Poisson noises are related to the Poisson random measure (see Gihman and Skorohod [10] or Hanson [12]) and are defined as 6 87 z L 7 d4 c e 4 L9 7 879 d c 7 (2) for O F to which consists of KML independent differentials of space time Poisson processes that are functions of the state, where is the ....

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103-162, 1996.

....[3, 1] that Work supported in part by the National Science Foundation Grant DMS 96 26692. This is a preprint of the regular paper published in the Proceedings of 1999 American Control Conference, 5 pages, June 1999 otherwise risk computational complexity due to a large number of variables [2]. A three stage stochastic optimal control problem in discrete time example is formulated in Section 3. The regular control for each stage is found in Section 4. The presence of white noise in both state and observations make an accurate estimation of the state more difficult. An accurate ....

F.B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," in Control and Dynamic Systems, vol. 76, Academic Press, pp. 103-162, April 1996.

.... expected value is determined by v(x; t f ) 1 2 x S f x for x 2 D x : 5) Stochastic dynamic programming is an application of the principle of optimality to the optimal expected performance value (4,3) and the general Ito chain rule for Markov stochastic processes in continuous time (see [1] for a tractable derivation) The partial differential equation of stochastic dynamic programming is known as the Hamilton Jacobi Bellman (HJB) equation and is subject to the final condition (5) which is given by 0 = v t (x; t) C 0 (x; t) i F 0 r x [v] j (x; t) 1 2 Gamma (G 0 ....

F. B. Hanson,"Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103-162, 1996.

.... but aside from a few specific problem forms, Bellman s curse of dimensionality in the state space limits its use in numerical methods to those with a relatively small number of state and control variables, though the use of high performance computing permits the treatment of larger state spaces [3]. The need to search the entire state space in dynamic programming, both for deterministic and stochastic problems, can lead to large scale computational demands. Thus the method has limited usefulness to applications such as reservoir management [6, 7] groundwater quality remediation [2, 4] and ....

....as reservoir management [6, 7] groundwater quality remediation [2, 4] and others. An alternate method, used by Kitanidis et al. 6, 7] for approximate solutions to the optimal control and cost to go performance index utilizes a stochastic perturbation of differential dynamic programming (DDP) [5, 3] to find an analytic solution of both the deterministic and caution (i.e. hedging or stochastic correction) terms. These terms are related to the interaction of estimation and control as found in dual control concepts [1] This stochastically perturbed DDP method gives a formally closed form ....

F.B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," in Control and Dynamic Systems, vol. 76, Academic Press, pp. 103-162, April 1996.

....system. The system is approximated in this paper by systematic perturbations due to small stochastic noise. The analysis of the problem is a variation on that used by Kitanidis and co workers [8, 9] for approximate solutions to the optimal control, utilizing differential dynamic programming (DDP) [5, 3] to find an analytic solution for the discrete time problem without searching the whole state space. However, calculations used here follow the systematic perturbations of the optimal control problem, with corrections, given in an earlier paper by Kern and Hanson [7] and are briefly reviewed in ....

....equations, as well. While the model used here has a limited number of variables, DDP lends itself to larger scale problems in applications such as reservoir management [8, 9] and groundwater quality remediation [4, 1] that otherwise risk computational complexity due to a large number of variables [3]. Work supported in part by the National Science Foundation Grant DMS 96 26692. The three stage stochastic optimal control problem in discrete time for the example in question is formulated in Section 3. The regular control for each stage is found in Section 4. The presence of white noise in ....

F.B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," in Control and Dynamic Systems, vol. 76, Academic Press, pp. 103-162, April 1996.

....critically depends on the mesh ratio of Deltat compared to some metric of Deltax and Deltay. For more information on the approximate quasi deterministic convergence criteria used, comparison to other methods, and additional references the reader is referred to the survey chapter of Hanson [26]. Figure 4 shows the optimal current value V (K; y; t) in million dollar units using optimal effort qE (K; y; t) r 1 versus a scaled price factor y Delta exp( Gammar 2 T ) i.e. with the deterministic inflationary part exp( r 2 T ) at the final time scaled out. The figure is intended to ....

F. B. Hanson, Techniques in computational stochastic dynamic programming, in Stochastic Digital Control System Techniques, within series Control and Dynamic Systems: Advances in Theory and Applications, Vol. 76, (C. T. Leondes, Editor), Academic Press, New York, NY, 1996, pp. 103-162.

....dynamic programming. Because of this, the computational requirements for the NLQGP problem, a search over the state space, is much greater than that for LQGP problem, following the evolution of the coefficient space. The NLQGP problem is similar to the LQG in control only model used by Hanson [11, 12] and Naimipour and Hanson [16] The main difference is that the NLQGP problem has a more robust representation of the Poisson noises. In [11, 16, 12] the Poisson noise used is a simpler form such that the Poisson coefficient depends only on the state, though nonlinearly; whereas the NLQGP problem ....

....that for LQGP problem, following the evolution of the coefficient space. The NLQGP problem is similar to the LQG in control only model used by Hanson [11, 12] and Naimipour and Hanson [16] The main difference is that the NLQGP problem has a more robust representation of the Poisson noises. In [11, 16, 12], the Poisson noise used is a simpler form such that the Poisson coefficient depends only on the state, though nonlinearly; whereas the NLQGP problem uses a compound Poisson noise that is bilinear in the control and a nonlinear function of the state. This bilinear term increases not only the ....

[Article contains additional citation context not shown here]

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," in Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp 103-162, April 1996.

....in terms of the state space. However, like stochastic dynamic programming, the computational requirements for the LQGP U problem, with a search over all the state space, are much greater than that for a pure LQGP problem. The LQGP U problem extends the LQGP in control only model used by Hanson [5, 6], and Naimipour and Hanson [9] by utilizing a more complex and realistic form for the Poisson processes. In this paper, we use a marked Poisson process so that the jump amplitudes or marks are random variables with an associated probability function that is independent of the arrival process. In ....

....k ) in which T i is the time of occurrence of the ith jump with amplitude M i (see, for example [10] This representation of the Poisson process provides more realism and flexibility for a wider range of stochastic control applications. The high performance techniques developed by Hanson et al. [5, 6, 9] for handling the large computational demands of the PDE of stochastic dynamic programming for Markovian problems in continuous time can be used for the marked Poisson process that we use here. Discrete jumps of random amplitude occur in the state because of the Poisson terms, which are ....

[Article contains additional citation context not shown here]

F. B. Hanson, "Techniques in Computational Stochastic Dynamic Programming," Digital and Control System Techniques and Applications, edited by C. T. Leondes, Academic Press, New York, pp. 103-162, 1996.

No context found.

Hanson, F.B., Techniques in Computational Stochastic Dynamic Programming, in Leondes, C.T., editor, Control and Dynamic Systems Volume 76: Stochastic Digital Control System Techniques, Academic Press, New York, pp. 103-162, 1996.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC