DMCA
Hybrid deterministic-stochastic methods for data fitting (2011)
| Citations: | 32 - 9 self |
BibTeX
@MISC{Friedlander11hybriddeterministic-stochastic,
author = {Michael P. Friedlander and Mark Schmidt},
title = {Hybrid deterministic-stochastic methods for data fitting},
year = {2011}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract. Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum. These methods can make great progress initially, but often slow as they approach a solution. In contrast, full-gradient methods achieve steady convergence at the expense of evaluating the full objective and gradient on each iteration. We explore hybrid methods that exhibit the benefits of both approaches. Rate-ofconvergence analysis shows that by controlling the sample size in an incremental gradient algorithm, it is possible to maintain the steady convergence rates of full-gradient methods. We detail a practical quasi-Newton implementation based on this approach. Numerical experiments illustrate its potential benefits. 1. Introduction. Data
Keyphrases
hybrid deterministic-stochastic method full-gradient method incremental gradient algorithm full objective inexpensive iteration potential benefit numerical experiment sample size optimization problem hybrid method steady convergence rate large number rate-ofconvergence analysis show data-fitting application steady convergence practical quasi-newton implementation great progress
