Citation Context

...ties, we obtain an additional convergence result without imposing stronger monotonicity properties on B, which requires averaging of the iterates. The present paper extends a short conference version =-=[47]-=- restricted to the minimization case. The paper is organized as follows. We first review related work in Section 2. Section 3 collects some preliminaries and Section 4 contains the main results of the...

Citation Context

...eded, and the Lipschitz property is essentially replaced with the condition ‖f(ξ, x)‖2 ≤ M(ξ)(1 + ‖x‖) where M(ξ) satisfies a moment condition. Regarding the convergence rate analysis, let us mention =-=[3, 35]-=- which investigate the performance of the algorithm xn+1 = proxγn+1g(xn − γn+1Hn+1) where Hn+1 is a noisy estimate of the gradient ∇f(xn). The same algorithm is addressed in [36] where the proximity o...

Citation Context

...1.5) The first instances of the stochastic iteration (1.5) can be traced back to [44] in the context of the gradient method, i.e., when A = 0 and B is the gradient of a convex function. Stochastic approximations in the gradient method were then investigated in the Russian literature of the late 1960s and early 1970s [27, 28, 29, 33, 42, 49]. Stochastic gradient methods have also been used extensively in adaptive signal processing, in control, and in machine learning, e.g., [3, 36, 54]. More generally, proximal stochastic gradient methods have been applied to various problems; see for instance [1, 26, 45, 48, 55]. The objective of the present paper is to provide an analysis of the stochastic forward-backward method in the context of Algorithm 1.3. Almost sure convergence of the iterates (xn)n∈N to a solution to Problem 1.1 will be established under general conditions on the sequences (un)n∈N, (an)n∈N, (γn)n∈N, and (λn)n∈N. In particular, a feature of our analysis is that it allows for relaxation parameters and it does not require that the proximal parameter sequence (γn)n∈N be vanishing. Our proofs are based on properties of stochastic quasi-Fejer iterations [18], for which we provide a novel converg...

Citation Context

...y. Inexact proximal gradient methods [17] allow to approximate the proximal gradients with controllable errors in a faster way while guaranteeing the convergence. Stochastic proximal gradient methods =-=[1, 15]-=- allow to compute the proximal gradients using a small set of data in a stochastic fashion while guaranteeing the convergence as well. Distributed proximal gradient methods [6] decompose the optimizat...