Results 1 
6 of
6
Approximate receding horizon approach for markov decision processes: average reward case
 Journal of Mathematical Analysis and Applications
, 2003
"... We consider an approximation scheme for solving Markov Decision Processes (MDPs) with countable state space, finite action space, and bounded rewards that uses an approximate solution of a fixed finitehorizon subMDP of a given infinitehorizon MDP to create a stationary policy, which we call “appr ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
(Show Context)
We consider an approximation scheme for solving Markov Decision Processes (MDPs) with countable state space, finite action space, and bounded rewards that uses an approximate solution of a fixed finitehorizon subMDP of a given infinitehorizon MDP to create a stationary policy, which we call “approximate receding horizon control”. We first analyze the performance of the approximate receding horizon control for infinitehorizon average reward under an ergodicity assumption, which also generalizes the result obtained by White [36]. We then study two examples of the approximate receding horizon control via lower bounds to the exact solution to the subMDP. The first control policy is based on a finitehorizon approximation of Howard’s policy improvement of a single policy and the second policy is based on a generalization of the single policy improvement for multiple policies. Along the study, we also provide a simple alternative proof on the policy improvement for countable state space. We finally discuss practical implementations of these schemes via simulation.
An Explicit Solution for the Value Function of a Priority Queue
 Queueing Systems
, 2004
"... In this paper we derive explicit expressions for the discounted expected weighted queuelengths and switching costs and for the longrun average version of this criterion in a multiclass preemptiveresume priority queue with Poisson arrivals and general service times. ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
In this paper we derive explicit expressions for the discounted expected weighted queuelengths and switching costs and for the longrun average version of this criterion in a multiclass preemptiveresume priority queue with Poisson arrivals and general service times.
Controlled jump Markov processes with local transitions and their fluid approximation
"... Abstract: Stochastic jump processes, especially birthanddeath processes, are widely used in the queuing theory, computer networks and information transmission. The state of such process describes the instant length of the queues (numbers of packets at different edges to be transmitted through the ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract: Stochastic jump processes, especially birthanddeath processes, are widely used in the queuing theory, computer networks and information transmission. The state of such process describes the instant length of the queues (numbers of packets at different edges to be transmitted through the net). If the birth and death rates are big, trajectories of such processes are close to the trajectories of deterministic dynamic systems. Therefore, if we consider the related optimal control problems, we expect that the optimal control strategy in the deterministic (‘fluid’) model will be nearly optimal in the underlying stochastic model. In the current paper, a new technique for calculating the accuracy of this approximation is described. In a nutshell, instead of the study of trajectories, we investigate the corresponding dynamic programming equations. It should be emphasized that we deal also with multipledimensional lattices, so that the results are applicable to complex communicating systems of queues. Other areas of application are population dynamics, mathematical epidemiology, and inventory systems.
OF PAGES
, 2001
"... ISR develops, applies and teaches advanced methodologies of design and analysis to solve complex, hierarchical, ..."
Abstract
 Add to MetaCart
(Show Context)
ISR develops, applies and teaches advanced methodologies of design and analysis to solve complex, hierarchical,
MARKOV GAMES: RECEDING HORIZON APPROACH
, 2001
"... ISR develops, applies and teaches advanced methodologies of design and analysis to solve complex, hierarchical, ..."
Abstract
 Add to MetaCart
(Show Context)
ISR develops, applies and teaches advanced methodologies of design and analysis to solve complex, hierarchical,