We consider open and closed multiclass queueing networks with Poisson arrivals (in open networks), exponentially distributed class dependent service times, and with class depen-dent deterministic or probabilistic routing. For open networks, the performance objective is to minimize, over all sequencing and routing policies, a weighted sum of the expected response times of different classes. Using a powerful technique involving quadratic or higher order potential functions, we propose variants of a method to derive polyhedral and non-linear spaces which contain the entire set of achievable response times under stable and preemptive scheduling policies. By optimizing over these spaces, we obtain lower bounds on achievable performance. In particular, we obtain a sequence of progressively more com-plicated nonlinear approximations (relaxations) which are progressively closer to the exact achievable space. In the special case of single station networks (multiclass queues and Klimov's model) and homogenous multiclass networks, our characterization gives exactly the achievable region. Consequently, the proposed method can be viewed as the natural

Devices]: Modes of Computation---Online computation General Terms: ALGORITHMS, THEORY Additional Key Words and Phrases: Stochastic scheduling, Approximation, Worst-case performance, Priority policy, LP-relaxation, WSEPT rule, Asymptotic optimality This research was partially supported by the German-Israeli Foundation for Scientific Research and Development (G.I.F.) under grant I 246-304.02/97. An extended abstract appeared in the Proceedings of the 2nd Int. Workshop on Approximation Algorithms for Combinatorial Optimization Problems (APPROX'99). Authors' addresses: Rolf H. Mohring and Marc Uetz. Technische Universitat Berlin, Fachbereich Mathematik, Sekr. MA 6--1, Straße des 17. Juni 136, 10623 Berlin, Germany, Email: fmoehring, uetzg@math.tu--berlin.de. Andreas S. Schulz. MIT, Sloan School of Management and Operations Research Center, E53--361, 30 Wadsworth St, Cambridge, MA 02139, Email: schulz@mit.edu. Permission to make digital or hard copies of part or all of this work for person...

We provide a review and synthesis of polyhedral approaches to machine scheduling problems. The choice of decision variables is the prime determinant of various formulations for such problems. Constraints, such as facet inducing inequalities for corresponding polyhedra, are often needed, in addition to those just required for the validity of the initial formulation, in order to obtain useful lower bounds and structural insights. We review formulations based on time–indexed variables; on linear ordering, start time and completion time variables; on assignment and positional date variables; and on traveling salesman variables. We point out relationship between various models, and provide a number of new results, as well as simplified new proofs of known results. In particular, we emphasize the important role that supermodular polyhedra and greedy algorithms play in many formulations and we analyze the strength of the lower and upper bounds obtained from different formulations and relaxations. We discuss separation algorithms for several classes of inequalities, and their potential applicability in generating cutting planes for the practical solution of such scheduling problems. We also review some recent results on approximation algorithms based on some of these formulations.

a primal-dual index heuristic

We show that if performance measures in a stochastic scheduling problem satisfy a set of so-called partial conservation laws (PCL), which extend previously studied generalized conservation laws (GCL), then the problem is solved optimally by a priority-index policy for an appropriate range of linear performance objectives, where the optimal indices are computed by a one-pass adaptive-greedy algorithm, based on Klimov's. We further apply this framework to investigate the indexability property of restless bandits introduced by Whittle, obtaining the following results: (1) we identify a class of restless bandits (PCL-indexable) which are indexable

Abstract not found

This paper develops a polyhedral approach to the design, analysis, and computation of dynamic allocation indices for scheduling binary-action (engage/rest) Markovian stochastic projects which can change state when rested (restless bandits (RBs)), based on partial conservation laws (PCLs). This extends previous work by the author [J. Nino-Mora (2001): Restless bandits, partial conservation laws and indexability. Adv. Appl. Probab. 33, 76--98], where PCLs were shown to imply the optimality of index policies with a postulated structure in stochastic scheduling problems, under admissible linear objectives, and they were deployed to obtain simple sufficient conditions for the existence of Whittle's (1988) RB index (indexability), along with an adaptive-greedy index algorithm. The new contributions include: (i) we develop the polyhedral foundation of the PCL framework, based on the structural and algorithmic properties of a new polytope associated with an accessible set system (J, (F-extended polymatroid); (ii) we present new dynamic allocation indices for RBs, motivated by an admission control model, which extend Whittle's and have a significantly increased scope; (iii) we deploy PCLs to obtain both sufficient conditions for the existence of the new indices (PCL-in- dexability), and a new adaptive-greedy index algorithm; (iv) we interpret PCL-indexability as a form of the classic economics law of diminishing marginal returns, and characterize the index as an optimal marginal cost rate; we further solve a related optimal constrained control problem; (v) we carry out a PCL-indexability analysis of the motivating admission control model, under time-discounted and long-run average criteria; this gives, under mild conditions, a new index characterization of optimal threshold...

the second author through the award of grants GR/K03043 and GR/M09308. We would also like to thank colleagues in the Department of Statistics, Newcastle University for their constructive comments on earlier drafts of the paper. The work of the third author was initiated during his stay at the Center for Operations Research and Econometrics (CORE) of the Universite catholique de Louvain, Belgium, where it was supported by EC individual Marie Curie Postdoctoral Fellowship no. ERBFMBICT961480. Further research support is acknowledged from Universitat Pompeu Fabra. The achievable region approach seeks solutions to stochastic optimisation problems by: (i) characterising the space of all possible performances (the achievable region) of the system of interest, and (ii) optimising the overall system-wide performance objective over this space. This is radically di erent from conventional formulations based on dynamic programming. The approach is explained with reference to a simple two-class queueing system. Powerful new methodologies due to the authors and co-workers are deployed to analyse a general multiclass queueing system with parallel servers and then to develop an approach to optimal load distribution across a network of interconnected stations. Finally, the approach is used for the rst time to analyse a class of intensity control problems.

We present the first approximation algorithms for a large class of budgeted learning problems. One classic example of the above is the budgeted multi-armed bandit problem. In this problem each arm of the bandit has an unknown reward distribution on which a prior is specified as input. The knowledge about the underlying distribution can be refined in the exploration phase by playing the arm and observing the rewards. However, there is a budget on the total number of plays allowed during exploration. After this exploration phase, the arm with the highest (posterior) expected reward is chosen for exploitation. The goal is to design the adaptive exploration phase subject to a budget constraint on the number of plays, in order to maximize the expected reward of the arm chosen for exploitation. While this problem is reasonably well understood in the infinite horizon setting or regret bounds, the budgeted version of the problem is NP-Hard. For this problem, and several generalizations, we provide approximate policies that achieve a reward within constant factor of the reward optimal policy. Our algorithms use a novel linear program rounding technique based on stochastic packing.

In this paper, we consider the restless bandit problem, which is one of the most well-studied generalizations of the celebrated stochastic multi-armed bandit problem in decision theory. In its ultimate generality, the restless bandit problem is known to be PSPACE-Hard to approximate to any non-trivial factor, and little progress has been made on this problem despite its significance in modeling activity allocation under uncertainty. We make progress on this problem by showing that for an interesting and general subclass that we term Monotone bandits, a surprisingly simple and intuitive greedy policy yields a factor 2 approximation. Such greedy policies are termed index policies, and are popular due to their simplicity and their optimality for the stochastic multi-armed bandit problem. The Monotone bandit problem strictly generalizes the stochastic multi-armed bandit problem, and naturally models multi-project scheduling where the state of a project becomes increasingly uncertain when the project is not scheduled. We develop several novel techniques in the design and analysis of the index policy. Our algorithm proceeds by introducing a novel “balance” constraint to the dual of a well-known LP relaxation to the restless bandit problem. This is followed by a structural characterization of the optimal solution by using both the exact primal as well as dual complementary slackness conditions. This yields an interpretation of the dual variables as potential functions from which we derive the index policy and the associated analysis. 1