Results 1  10
of
17
On the complexity of bandit and derivativefree stochastic convex optimization
 CoRR
"... The problem of stochastic convex optimization with bandit feedback (in the learning community) or without knowledge of gradients (in the optimization community) has received much attention in recent years, in the form of algorithms and performance upper bounds. However, much less is known about the ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
The problem of stochastic convex optimization with bandit feedback (in the learning community) or without knowledge of gradients (in the optimization community) has received much attention in recent years, in the form of algorithms and performance upper bounds. However, much less is known about the inherent complexity of these problems, and there are few lower bounds in the literature, especially for nonlinear functions. In this paper, we investigate the attainable error/regret in the bandit and derivativefree settings, as a function of the dimension d and the available number of queries T. We provide a precise characterization of the attainable performance for stronglyconvex and smooth functions, which also imply a nontrivial lower bound for more general problems. Moreover, we prove that in both the bandit and derivativefree setting, the required number of queries must scale at least quadratically with the dimension. Finally, we show that on the natural class of quadratic functions, it is possible to obtain a “fast ” O(1/T) error rate in terms of T, under mild assumptions, even without having access to gradients. To the best of our knowledge, this is the first such rate in a derivativefree stochastic setting, and holds despite previous results which seem to imply the contrary.
Algorithmic Connections between Active Learning and Stochastic Convex Optimization
"... Abstract. Interesting theoretical associations have been established by recent papers between the fields of active learning and stochastic convex optimization due to the common role of feedback in sequential querying mechanisms. In this paper, we continue this thread in two parts by exploiting these ..."
Abstract

Cited by 6 (4 self)
 Add to MetaCart
(Show Context)
Abstract. Interesting theoretical associations have been established by recent papers between the fields of active learning and stochastic convex optimization due to the common role of feedback in sequential querying mechanisms. In this paper, we continue this thread in two parts by exploiting these relations for the first time to yield novel algorithms in both fields, further motivating the study of their intersection. First, inspired by a recent optimization algorithm that was adaptive to unknown uniform convexity parameters, we present a new active learning algorithm for onedimensional thresholds that can yield minimax rates by adapting to unknown noise parameters. Next, we show that one can perform ddimensional stochastic minimization of smooth uniformly convex functions when only granted oracle access to noisy gradient signs along any coordinate instead of realvalued gradients, by using a simple randomized coordinate descent procedure where each line search can be solved by 1dimensional active learning, provably achieving the same error convergence rate as having the entire realvalued gradient. Combining these two parts yields an algorithm that solves stochastic convex optimization of uniformly convex and smooth functions using only noisy gradient signs by repeatedly performing active learning, achieves optimal rates and is adaptive to all unknown convexity and smoothness parameters. 1
Informationtheoretic lower bounds for convex optimization with erroneous oracles
"... Abstract We consider the problem of optimizing convex and concave functions with access to an erroneous zerothorder oracle. In particular, for a given function x → f (x) we consider optimization when one is given access to absolute error oracles that return values in [f (x) − , f (x) + ] or relati ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Abstract We consider the problem of optimizing convex and concave functions with access to an erroneous zerothorder oracle. In particular, for a given function x → f (x) we consider optimization when one is given access to absolute error oracles that return values in [f (x) − , f (x) + ] or relative error oracles that return value in , for some > 0. We show stark information theoretic impossibility results for minimizing convex functions and maximizing concave functions over polytopes in this model.
A COLLABORATIVE 20 QUESTIONS MODEL FOR TARGET SEARCH WITH HUMANMACHINE INTERACTION
"... We consider the problem of 20 questions with noise for collaborative players under the minimum entropy criterion [1] in the setting of stochastic search, with application to target localization. First, assuming conditionally independent collaborators, we characterize the structure of the optimal pol ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
We consider the problem of 20 questions with noise for collaborative players under the minimum entropy criterion [1] in the setting of stochastic search, with application to target localization. First, assuming conditionally independent collaborators, we characterize the structure of the optimal policy for constructing the sequence of questions. This generalizes the single player probabilistic bisection method [1, 2] for stochastic search problems. Second, we prove a separation theorem showing that optimal joint queries achieve the same performance as a greedy sequential scheme. Third, we establish convergence rates of the meansquare error (MSE). Fourth, we derive upper bounds on the MSE of the sequential scheme. This framework provides a mathematical model for incorporating a human in the loop for active machine learning systems. Index Terms — optimal query selection, humanmachine interaction, target localization, convergence rate, minimum entropy. 1.
On ZerothOrder Stochastic Convex Optimization via Random Walks
, 2014
"... We propose a method for zeroth order stochastic convex optimization that attains the suboptimality rate of Õ(n7T−1/2) after T queries for a convex bounded function f: Rn → R. The method is based on a random walk (the Ball Walk) on the epigraph of the function. The randomized approach circumvents t ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
We propose a method for zeroth order stochastic convex optimization that attains the suboptimality rate of Õ(n7T−1/2) after T queries for a convex bounded function f: Rn → R. The method is based on a random walk (the Ball Walk) on the epigraph of the function. The randomized approach circumvents the problem of gradient estimation, and appears to be less sensitive to noisy function evaluations compared to noiseless zeroth order methods.
Beating Bandits in Gradually Evolving Worlds ChaoKai Chiang1;2
"... Consider the online convex optimization problem, in which a player has to choose actions iteratively and suffers corresponding losses according to some convex loss functions, and the goal is to minimize the regret. In the fullinformation setting, the player after choosing her action can observe th ..."
Abstract
 Add to MetaCart
(Show Context)
Consider the online convex optimization problem, in which a player has to choose actions iteratively and suffers corresponding losses according to some convex loss functions, and the goal is to minimize the regret. In the fullinformation setting, the player after choosing her action can observe the whole loss function in that round, while in the bandit setting, the only information the player can observe is the loss value of that action. Designing such bandit algorithms appears challenging, as the best regret currently achieved for general convex loss functions is much higher than that in the fullinformation setting, while for strongly convex loss functions, there is even a regret lower bound which is exponentially higher than that achieved in the fullinformation setting. To aim for smaller regrets, we adopt a relaxed twopoint bandit setting in which the player can play two actions in each round and observe the loss values of those two actions. Moreover, we consider loss functions parameterized by their deviation D, which measures how fast they evolve, and we study how regrets depend on D. We show that twopoint bandit algorithms can in fact achieve regrets matching those in the fullinformation setting in terms of D. More precisely, for convex loss functions, we achieve a regret of O( D), while for strongly convex loss functions, we achieve a regret of O(lnD), which is much smaller than the Ω( D) lower bound in the traditional bandit setting.
Simple Complexity Analysis of Direct Search
"... Abstract We consider the problem of unconstrained minimization of a smooth function in the derivativefree setting. In particular, we study the direct search method (of directional type). Despite relevant research activity spanning several decades, until recently no complexity guaranteesbounds on th ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract We consider the problem of unconstrained minimization of a smooth function in the derivativefree setting. In particular, we study the direct search method (of directional type). Despite relevant research activity spanning several decades, until recently no complexity guaranteesbounds on the number of function evaluations needed to find a satisfying pointfor methods of this type were established. Moreover, existing complexity results require long proofs and the resulting bounds have a complicated form. In this paper we give a very brief and insightful analysis of direct search for nonconvex, convex and strongly convex objective function, based on the observation that what is in the literature called an "unsuccessful step", is in fact a step that can drive the analysis. We match the existing results in their dependence on the problem dimension (n) and error tolerance ( ), but the overall complexity bounds are much simpler, easier to interpret, and have better dependence on other problem parameters. In particular, we show that the number of function evaluations needed to find an solution is O(n 2 / ) (resp. O(n 2 log(1/ ))) for the problem of minimizing a convex (resp. strongly convex) smooth function. In the nonconvex smooth case, the bound is O(n 2 / 2 ), with the goal being the reduction of the norm of the gradient below .
Computational and statistical advances in testing and learning
, 2015
"... This thesis makes fundamental computational and statistical advances in testing and estimation, making critical progress in theory and application of classical statistical methods like classification, regression and hypothesis testing, and understanding the relationships between them. Our work con ..."
Abstract
 Add to MetaCart
This thesis makes fundamental computational and statistical advances in testing and estimation, making critical progress in theory and application of classical statistical methods like classification, regression and hypothesis testing, and understanding the relationships between them. Our work connects multiple fields in often counterintuitive and surprising ways, leading to new theory, new algorithms, and new insights, and ultimately to a crossfertilization of varied fields like optimization, statistics and machine learning. The first of three thrusts has to do with active learning, a form of sequential learning from feedbackdriven queries that often has a provable statistical advantage over passive learning. We unify concepts from two seemingly different areas — active learning and stochastic firstorder optimization. We use this unified view to develop new lower bounds for stochastic optimization using tools from active learning and new algorithms for active learning using ideas from optimization. We also study the effect of feature noise, or errorsinvariables, on