Results 1  10
of
16
Query Complexity of DerivativeFree Optimization
"... This paper provides lower bounds on the convergence rate of Derivative Free Optimization (DFO) with noisy function evaluations, exposing a fundamental and unavoidable gap between the performance of algorithms with access to gradients and those with access to only function evaluations. However, there ..."
Abstract

Cited by 17 (0 self)
 Add to MetaCart
(Show Context)
This paper provides lower bounds on the convergence rate of Derivative Free Optimization (DFO) with noisy function evaluations, exposing a fundamental and unavoidable gap between the performance of algorithms with access to gradients and those with access to only function evaluations. However, there are situations in which DFO is unavoidable, and for such situations we propose a new DFO algorithm that is proved to be near optimal for the class of strongly convex objective functions. A distinctive feature of the algorithm is that it uses only Booleanvalued function comparisons, rather than function evaluations. This makes the algorithm useful in an even wider range of applications, such as optimization based on paired comparisons from human subjects, for example. We also show that regardless of whether DFO is based on noisy function evaluations or Booleanvalued function comparisons, the convergence rate is the same. 1
On the complexity of bandit and derivativefree stochastic convex optimization
 CoRR
"... The problem of stochastic convex optimization with bandit feedback (in the learning community) or without knowledge of gradients (in the optimization community) has received much attention in recent years, in the form of algorithms and performance upper bounds. However, much less is known about the ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
The problem of stochastic convex optimization with bandit feedback (in the learning community) or without knowledge of gradients (in the optimization community) has received much attention in recent years, in the form of algorithms and performance upper bounds. However, much less is known about the inherent complexity of these problems, and there are few lower bounds in the literature, especially for nonlinear functions. In this paper, we investigate the attainable error/regret in the bandit and derivativefree settings, as a function of the dimension d and the available number of queries T. We provide a precise characterization of the attainable performance for stronglyconvex and smooth functions, which also imply a nontrivial lower bound for more general problems. Moreover, we prove that in both the bandit and derivativefree setting, the required number of queries must scale at least quadratically with the dimension. Finally, we show that on the natural class of quadratic functions, it is possible to obtain a “fast ” O(1/T) error rate in terms of T, under mild assumptions, even without having access to gradients. To the best of our knowledge, this is the first such rate in a derivativefree stochastic setting, and holds despite previous results which seem to imply the contrary.
Escaping the local minima via simulated annealing: Optimization of approximately convex functions
, 2015
"... We consider the problem of optimizing an approximately convex function over a bounded convex set in Rn using only function evaluations. The problem is reduced to sampling from an approximately logconcave distribution using the HitandRun method, which is shown to have the same O ∗ complexity as s ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
We consider the problem of optimizing an approximately convex function over a bounded convex set in Rn using only function evaluations. The problem is reduced to sampling from an approximately logconcave distribution using the HitandRun method, which is shown to have the same O ∗ complexity as sampling from logconcave distributions. In addition to extend the analysis for logconcave distributions to approximate logconcave distributions, the implementation of the 1dimensional sampler of the HitandRun walk requires new methods and analysis. The algorithm then is based on simulated annealing which does not relies on first order conditions which makes it essentially immune to local minima. We then apply the method to different motivating problems. In the context of zeroth order stochastic convex optimization, the proposed method produces an minimizer after O∗(n7.5−2) noisy function evaluations by inducing a O(/n)approximately log concave distribution. We also consider in detail the case when the “amount of nonconvexity ” decays towards the optimum of the function. Other applications of the method discussed in this work include private computation of empirical risk minimizers, twostage stochastic programming, and approximate dynamic programming for online learning. 1.
Informationtheoretic lower bounds for convex optimization with erroneous oracles
"... Abstract We consider the problem of optimizing convex and concave functions with access to an erroneous zerothorder oracle. In particular, for a given function x → f (x) we consider optimization when one is given access to absolute error oracles that return values in [f (x) − , f (x) + ] or relati ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Abstract We consider the problem of optimizing convex and concave functions with access to an erroneous zerothorder oracle. In particular, for a given function x → f (x) we consider optimization when one is given access to absolute error oracles that return values in [f (x) − , f (x) + ] or relative error oracles that return value in , for some > 0. We show stark information theoretic impossibility results for minimizing convex functions and maximizing concave functions over polytopes in this model.
Nonstationary Stochastic Optimization
, 2013
"... We consider a nonstationary variant of a sequential stochastic optimization problem, where the underlying cost functions may change along the horizon. We propose a measure, termed variation budget, that controls the extent of said change, and study how restrictions on this budget impact achievable ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We consider a nonstationary variant of a sequential stochastic optimization problem, where the underlying cost functions may change along the horizon. We propose a measure, termed variation budget, that controls the extent of said change, and study how restrictions on this budget impact achievable performance. We identify sharp conditions under which it is possible to achieve longrunaverage optimality and more refined performance measures such as rate optimality that fully characterize the complexity of such problems. In doing so, we also establish a strong connection between two rather disparate strands of literature: adversarial online convex optimization; and the more traditional stochastic approximation paradigm (couched in a nonstationary setting). This connection is the key to deriving well performing policies in the latter, by leveraging structure of optimal policies in the former. Finally, tight bounds on the minimax regret allow us to quantify the “price of nonstationarity, ” which mathematically captures the added complexity embedded in a temporally changing environment versus a stationary one.
Bandits with knapsacks: Dynamic procurement for crowdsourcing
 In The 3rd Workshop on Social Computing and User Generated Content, colocated with ACM EC
, 2013
"... Abstract In a basic version of the dynamic procurement problem, the algorithm has a budget B to spend, and is facing n agents (potential sellers) that are arriving sequentially. The algorithm offers a takeitorleaveit price to each arriving seller; the sellers value for an item is an independent ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Abstract In a basic version of the dynamic procurement problem, the algorithm has a budget B to spend, and is facing n agents (potential sellers) that are arriving sequentially. The algorithm offers a takeitorleaveit price to each arriving seller; the sellers value for an item is an independent sample from some fixed (but unknown) distribution. The goal is to maximize the number of items bought. This problem is particularly relevant to the emerging domain of crowdsourcing, where agents correspond to the (relatively inexpensive) workers on a crowdsourcing platform such as Amazon Mechanical Turk, and "items" bought/sold correspond to simple jobs ("microtasks") that can be performed by these workers. The algorithm corresponds to the "client": an entity that submits jobs and benefits from them being completed. The basic formulation admits various generalizations, e.g. to multiple job types. We also address an alternative model in which the requester posts offers to the entire crowd. We model the dynamic procurement problems as multiarmed bandit problems with a budget constraint. We define "bandits with knapsacks": a broad class of multiarmed bandit problems with knapsackstyle resourceutilization constraints which subsumes dynamic procurement and a host of other applications. A distinctive feature of our problem, in comparison to the existing regretminimization literature, is that the optimal policy for a given latent distribution may significantly outperform the policy that plays the optimal fixed arm. Consequently, achieving sublinear regret in the banditswithknapsacks problem is significantly more challenging than in conventional bandit problems. Our main result is an algorithm for a version of banditswithknapsacks with finitely many possible actions. It is a primaldual algorithm with multiplicative updates; the regret of this algorithm is close to the informationtheoretic optimum. We derive corollaries for dynamic procurement using uniform discretization of prices. * This is a refocused and shortened version of a paper which is under submission. That paper, titled "Bandits with Knapsacks"
Multiscale exploration of convex functions and bandit convex optimization Microsoft Research
, 2016
"... Abstract We construct a new map from a convex function to a distribution on its domain, with the property that this distribution is a multiscale exploration of the function. We use this map to solve a decadeold open problem in adversarial bandit convex optimization by showing that the minimax regr ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract We construct a new map from a convex function to a distribution on its domain, with the property that this distribution is a multiscale exploration of the function. We use this map to solve a decadeold open problem in adversarial bandit convex optimization by showing that the minimax regret for this problem is O(poly(n) √ T ), where n is the dimension and T the number of rounds. This bound is obtained by studying the dual Bayesian maximin regret via the information ratio analysis of Russo and Van Roy, and then using the multiscale exploration to construct a new algorithm for the Bayesian convex bandit problem.
Finite sample convergence rates of zeroorder stochastic optimization methods
 In Advances in Neural Information Processing Systems 25
, 2012
"... Abstract We consider derivativefree algorithms for stochastic optimization problems that use only noisy function values rather than gradients, analyzing their finitesample convergence rates. We show that if pairs of function values are available, algorithms that use gradient estimates based on ra ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract We consider derivativefree algorithms for stochastic optimization problems that use only noisy function values rather than gradients, analyzing their finitesample convergence rates. We show that if pairs of function values are available, algorithms that use gradient estimates based on random perturbations suffer a factor of at most √ d in convergence rate over traditional stochastic gradient methods, where d is the problem dimension. We complement our algorithmic development with informationtheoretic lower bounds on the minimax convergence rate of such problems, which show that our bounds are sharp with respect to all problemdependent quantities: they cannot be improved by more than constant factors.
On ZerothOrder Stochastic Convex Optimization via Random Walks
, 2014
"... We propose a method for zeroth order stochastic convex optimization that attains the suboptimality rate of Õ(n7T−1/2) after T queries for a convex bounded function f: Rn → R. The method is based on a random walk (the Ball Walk) on the epigraph of the function. The randomized approach circumvents t ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
We propose a method for zeroth order stochastic convex optimization that attains the suboptimality rate of Õ(n7T−1/2) after T queries for a convex bounded function f: Rn → R. The method is based on a random walk (the Ball Walk) on the epigraph of the function. The randomized approach circumvents the problem of gradient estimation, and appears to be less sensitive to noisy function evaluations compared to noiseless zeroth order methods.