Results 1  10
of
46
Sequencing and routing in multiclass queueing networks part I: Feedback regulation
 SIAM J. Control Optim
"... Abstract. Part II continues the development of policy synthesis techniques for multiclass queueing networks based upon a linear fluid model. The following are shown: (i) A relaxation of the fluid model based on workload leads to an optimization problem of lower dimension. An analogous workloadrelax ..."
Abstract

Cited by 55 (12 self)
 Add to MetaCart
(Show Context)
Abstract. Part II continues the development of policy synthesis techniques for multiclass queueing networks based upon a linear fluid model. The following are shown: (i) A relaxation of the fluid model based on workload leads to an optimization problem of lower dimension. An analogous workloadrelaxation is introduced for the stochastic model. These relaxed control problems admit pointwise optimal solutions in many instances. (ii) A translation to the original fluid model is almost optimal, with vanishing relative error as the networkload ρ approaches one. It is pointwise optimal after a short transient period, provided a pointwise optimal solution exists for the relaxed control problem. (iii) A translation of the optimal policy for the fluid model provides a policy for the stochastic networkmodel that is almost optimal in heavy traffic, over all solutions to the relaxed stochastic model, again with vanishing relative error. The regret is of order  log(1 − ρ).
A CostShaping Linear Program for AverageCost Approximate Dynamic Programming with Performance Guarantees
, 2006
"... ..."
Dynamic SafetyStocks for Asymptotic Optimality in Stochastic Networks
 Queueing Syst. Theory Appl
, 2004
"... This paper concerns control of stochastic networks using statedependent safetystocks. Three examples are considered: a pair of tandem queues; a simple routing model; and the DaiWang reentrant line. In each case, a single policy is proposed that is independent of network load # . ..."
Abstract

Cited by 16 (7 self)
 Add to MetaCart
(Show Context)
This paper concerns control of stochastic networks using statedependent safetystocks. Three examples are considered: a pair of tandem queues; a simple routing model; and the DaiWang reentrant line. In each case, a single policy is proposed that is independent of network load # .
Workload Models for Stochastic Networks: Value Functions and Performance Evaluation
, 2005
"... This paper concerns control and performance evaluation for stochastic network models. Structural properties of value functions are developed for controlled Brownian motion (CBM) and deterministic (fluid) workloadmodels, leading to the following conclusions: Outside of a nullset of network paramete ..."
Abstract

Cited by 16 (9 self)
 Add to MetaCart
This paper concerns control and performance evaluation for stochastic network models. Structural properties of value functions are developed for controlled Brownian motion (CBM) and deterministic (fluid) workloadmodels, leading to the following conclusions: Outside of a nullset of network parameters, (i) The fluid valuefunction is a smooth function of the initial state. Under further minor conditions, the fluid valuefunction satisfies the derivative boundary conditions that are required to ensure it is in the domain of the extended generator for the CBM model. Exponential ergodicity of the CBM model is demonstrated as one consequence. (ii) The fluid valuefunction provides a shadow function for use in simulation variance reduction for the stochastic model. The resulting simulator satisfies an exact large deviation principle, while a standard simulation algorithm does not satisfy any such bound. (iii) The fluid valuefunction provides upper and lower bounds on performance for the CBM model. This follows from an extension of recent linear programming approaches to performance evaluation.
Stability and asymptotic optimality of generalized MaxWeight policies
 SIAM J. CONTROL AND OPT
, 2007
"... It is shown that stability of the celebrated MaxWeight or back pressure policies is a consequence of the following interpretation: either policy is myopic with respect to a surrogate value function of a very special form, in which the “marginal disutility ” at a buffer vanishes for vanishingly small ..."
Abstract

Cited by 16 (2 self)
 Add to MetaCart
(Show Context)
It is shown that stability of the celebrated MaxWeight or back pressure policies is a consequence of the following interpretation: either policy is myopic with respect to a surrogate value function of a very special form, in which the “marginal disutility ” at a buffer vanishes for vanishingly small buffer population. This observation motivates the hMaxWeight policy, defined for a wide class of functions h. These policies share many of the attractive properties of the MaxWeight policy: (i) Arrival rate data is not required in the policy. (ii) Under a variety of general conditions, the policy is stabilizing when h is a perturbation of a monotone linear function, a monotone quadratic, or a monotone Lyapunov function for the fluid model. (iii) A perturbation of the relative value function for a workload relaxation gives rise to a myopic policy that is approximately averagecost optimal in heavy traffic, with logarithmic regret. The first results are obtained for a general Markovian network model. Asymptotic optimality is established for a general Markovian scheduling model with a single bottleneck, and homogeneous servers.
Large deviation asymptotics and control variates for simulating large functions
, 2005
"... Consider the normalized partial sums of a realvalued function F of a Markov chain, φn: = n −1 n−1 F(Φ(k)), n ≥ 1. k=0 The chain {Φ(k) : k ≥ 0} takes values in a general state space X, with transition kernel P, and it is assumed that the Lyapunov drift condition holds: PV ≤ V −W +bIC where V: X → (0 ..."
Abstract

Cited by 16 (6 self)
 Add to MetaCart
(Show Context)
Consider the normalized partial sums of a realvalued function F of a Markov chain, φn: = n −1 n−1 F(Φ(k)), n ≥ 1. k=0 The chain {Φ(k) : k ≥ 0} takes values in a general state space X, with transition kernel P, and it is assumed that the Lyapunov drift condition holds: PV ≤ V −W +bIC where V: X → (0, ∞), W: X → [1, ∞), the set C is small, and W dominates F. Under these assumptions, the following conclusions are obtained: (i) It is known that this drift condition is equivalent to the existence of a unique invariant distribution π satisfying π(W) < ∞, and the Law of Large Numbers holds for any function F dominated by W: φn → φ: = π(F), a.s., n → ∞. (ii) The lower error probability defined by P{φn ≤ c}, for c < φ, n ≥ 1, satisfies a large deviation limit theorem when the function F satisfies a monotonicity condition. Under additional minor conditions an exact large deviations expansion is obtained. (iii) If W is nearmonotone then controlvariates are constructed based on the Lyapunov function V, providing a pair of estimators that together satisfy nontrivial large asymptotics for the lower and upper error probabilities. In an application to simulation of queues it is shown that exact large deviation asymptotics are possible even when the estimator does not satisfy a Central Limit Theorem.
Positive Harris Recurrence and Diffusion Scale Analysis of a Push Pull Queueing Network
, 2009
"... We consider a push pull queueing network with two servers and two types of jobs which are processed by the two servers in opposite order, with stochastic generally distributed processing times. This push pull network was introduced by Kopzon and Weiss, who assumed exponential processing times. It is ..."
Abstract

Cited by 15 (11 self)
 Add to MetaCart
We consider a push pull queueing network with two servers and two types of jobs which are processed by the two servers in opposite order, with stochastic generally distributed processing times. This push pull network was introduced by Kopzon and Weiss, who assumed exponential processing times. It is similar to the KumarSeidman RybkoStolyar (KSRS) multiclass queueing network, with the distinction that instead of random arrivals, there is an infinite supply of jobs of both types. Unlike the KSRS network, we can find policies under which our push pull network works at full utilization, with both servers busy at all times, and without being congested. We perform fluid and diffusion scale analysis of this network under such policies, to show fluid stability, positive Harris recurrence, and to obtain a diffusion limit for the network. On the diffusion scale the network is empty, and the departures of the two types of jobs are highly negatively correlated Brownian motions. Using similar methods we also derive a diffusion limit of a reentrant line with infinite supply of work.
Approximate Dynamic Programming using Fluid and Diffusion Approximations with Applications to Power Management
"... Abstract—TD learning and its refinements are powerful tools for approximating the solution to dynamic programming problems. However, the techniques provide the approximate solution only within a prescribed finitedimensional function class. Thus, the question that always arises is how should the fun ..."
Abstract

Cited by 11 (5 self)
 Add to MetaCart
(Show Context)
Abstract—TD learning and its refinements are powerful tools for approximating the solution to dynamic programming problems. However, the techniques provide the approximate solution only within a prescribed finitedimensional function class. Thus, the question that always arises is how should the function class be chosen? The goal of this paper is to propose an approach for TD learning based on choosing the function class using the solutions to associated fluid and diffusion approximations. In order to illustrate this new approach, the paper focuses on an application to dynamic speed scaling for power management.
Management of DemandDriven Production Systems
 IEEE TRANS. AUTOMAT. CONTROL
, 2004
"... Controlsynthesis techniques are developed for demand driven production systems. The resulting policies are timeoptimal for a deterministic model, and approximately timeoptimal for a stochastic model. Moreover, they are easily adapted to take into account a range of issues that arise in a realisti ..."
Abstract

Cited by 10 (6 self)
 Add to MetaCart
Controlsynthesis techniques are developed for demand driven production systems. The resulting policies are timeoptimal for a deterministic model, and approximately timeoptimal for a stochastic model. Moreover, they are easily adapted to take into account a range of issues that arise in a realistic, dynamic environment. In particular, control synthesis techniques are developed for models in which resources are temporarily unavailable. This may be due to failure, maintenance, or an unanticipated change in demand. These conclusions are based upon the following development...
Adaptive control variates for finitehorizon simulation
 Math. Oper. Res
, 2007
"... Adaptive Monte Carlo methods are simulation efficiency improvement techniques designed to adaptively tune simulation estimators. Most of the work on adaptive Monte Carlo methods has been devoted to adaptively tuning importance sampling schemes. We instead focus on adaptive methods based on control ..."
Abstract

Cited by 10 (2 self)
 Add to MetaCart
Adaptive Monte Carlo methods are simulation efficiency improvement techniques designed to adaptively tune simulation estimators. Most of the work on adaptive Monte Carlo methods has been devoted to adaptively tuning importance sampling schemes. We instead focus on adaptive methods based on control variate schemes. We introduce two adaptive control variate methods, and develop their asymptotic properties. The first method uses stochastic approximation to adaptively tune control variate estimators. It is easy to implement, but it requires some nontrivial tuning of parameters. The second method is based on sample average approximation. Tuning is no longer required, but it can be computationally expensive. Numerical results for the pricing of barrier options are presented to demonstrate the methods. 1