Stochastic first order methods in smooth convex optimization (2011)
| Citations: | 7 - 0 self |
BibTeX
@MISC{Devolder11stochasticfirst,
author = {Olivier Devolder},
title = {Stochastic first order methods in smooth convex optimization},
year = {2011}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
In this paper, we are interested in the development of efficient first-order methods for convex optimization problems in the simultaneous presence of smoothness of the objective function and stochasticity in the first-order information. First, we consider the Stochastic Primal Gradient method, which is nothing else but the Mirror Descent SA method applied to a smooth function and we develop new practical and efficient stepsizes policies. Based on the machinery of estimates sequences functions, we develop also two new methods, a Stochastic Dual Gradient Method and an accelerated Stochastic Fast Gradient Method. Convergence rates on average, probabilities of large deviations and accuracy certificates are studied. All of these methods are designed in order to decrease the effect of the stochastic noise at an unimprovable rate and to be easily implementable in practice (the practical efficiency of our method is confirmed by numerical experiments). Furthermore, the biased case, when the oracle is not
Keyphrases
first order algorithm nuclear norm minimization core discussion paper smooth convex optimization stochastic first order method biased case new method convergence rate stochastic primal gradient method stochastic noise accelerated stochastic fast gradient method large deviation first-order information efficient first-order method mirror descent sa method simultaneous presence smooth function efficient stepsizes policy estimate sequence function unimprovable rate practical efficiency accuracy certificate objective function numerical experiment stochastic dual gradient method convex optimization problem
