DMCA
Two-stage index computation for bandits with switching penalties I: switching costs
| Venue: | Working Paper 07-41, Statistics and Econometrics Series 09, http://halweb.uc3m.es/jnino/eng/public2.html, Univ. Carlos III de |
| Citations: | 1 - 1 self |
BibTeX
@INPROCEEDINGS{Niño-mora_two-stageindex,
author = {José Niño-mora},
title = {Two-stage index computation for bandits with switching penalties I: switching costs},
booktitle = {Working Paper 07-41, Statistics and Econometrics Series 09, http://halweb.uc3m.es/jnino/eng/public2.html, Univ. Carlos III de},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
This paper addresses the multi-armed bandit problem with switching penalties including both costs and delays, extending results of the companion paper [J. Niño-Mora. ``Two-Stage Index Computation for Bandits with Switching Penalties I: Switching Costs.' ' Conditionally accepted at INFORMS J. Comp.], which addressed the no switching delays case. Asawa and Teneketzis (1996) introduced an index for bandits with delays that partly characterizes optimal policies, attaching to each bandit state a ``continuation index' ' (its Gittins index) and a ``switching index,'' yet gave no algorithm for it. This paper presents an efficient, decoupled computation method, which in a first stage computes the continuation index and then, in a second stage, computes the switching index an order of magnitude faster in at most arithmetic operations for an-state bandit. The paper exploits the fact that the Asawa and Teneketzis index is the Whittle, or marginal productivity, index of a classic bandit with switching penalties in its semi-Markov restless reformulation, by deploying work-reward analysis and LP-indexability methods introduced by the author. A computational study demonstrates the dramatic runtime savings achieved by the new algorithm, the near-optimality of the index policy, and its substantial gains
Keyphrases
two-stage index computation switching cost continuation index an-state bandit gittins index computational study optimal policy classic bandit work-reward analysis index policy teneketzis index substantial gain switching penalty multi-armed bandit problem companion paper semi-markov restless reformulation bandit state dramatic runtime saving arithmetic operation lp-indexability method decoupled computation method second stage marginal productivity new algorithm first stage
