Citation Context

...ed without requiring the subproblems to have unique solutions [6]. Recently Razaviyayn et al [15] have shown that the BCD converges if each subproblem (2) is solved inexactly, by way of optimizing certain surrogate functions. Luo and Tseng in [10] have shown that when problem (1) satisfies certain additional assumptions such as having a smooth composite objective and a polyhedral feasible set, then BCD converges linearly without requiring the objective to be strongly convex. There are many recent works on showing iteration complexity for randomizedBCGD (block coordinate gradient descent), see [17, 12, 8, 16, 9] and the references therein. However the results on the classical cyclic BCD is rather scant. Saha and Tewari [18] show that the cyclic BCD achieves sublinear convergence for a family of special LASSO problems. Nutini et al [13] show that when the problem is strongly convex, unconstrained and smooth, BCGD with certain Gauss-Southwell block selection rule could be faster than the randomized rule. Recently Beck and Tetruashvili show that cyclic BCGD converges sublinearly if the objective is smooth. Subsequently Hong et al in [7] show that such sublinear rate not only can be extended to problems ...

Citation Context

... , R}. Clearly L ≤ 1/N ∑N i=1 Li, and the equality can be achieved in the worst case. For simplicity of analysis we will further assume that L0 ≤ 1N ∑N i=1 Li. A-(c) Each ηi in (1.3) satisfies ηi > Li/N ; if g0 is nonconvex, then ∑N i=1 ηi > 3L0. Assumption A-(c) implies that L (x, z;λ) is strongly convex w.r.t. each xi and z, with modulus γi := ηi − Li/N and γz = ∑N i=1 ηi − L0, respectively [27, Theorem 2.1]. We then define the prox-gradient (pGRAD) for (1.1), which will serve as a measure of stationarity. It can be checked that the pGRAD vanishes at the set of stationary solutions of (1.1) [20]. Definition 1.1. The proximal gradient of problem (1.1) is given by (for any γ > 0) ∇fγ(z) := γ ( z − proxγp+ιZ [z − 1/γ∇(g(z) + g0(z))] ) , with proxγp+ιZ [u] := argmin u∈Z p(u)+ γ 2 ‖z−u‖2. 2 The NESTT-G Algorithm Algorithm Description. We present a primal-dual splitting scheme for the reformulated problem (1.2). The algorithm is referred to as the NESTT with Gradient step (NESTT-G) since each agent only requires to know the gradient of each component function. To proceed, let us define the following function (for some constants {αi > 0}Ni=1): Vi(xi, z;λi) = 1 N gi(z) + 1 N 〈∇gi(z), xi − z...

Citation Context

...our approach extends also to parallel, random schemes. Feature 3: The list of works dealing with BCD methods for nonconvex F ’s is short: [22], [29] for random sequential methods; and [7], [17]–[19], =-=[38]-=- for deterministic parallel ones. The only (very recent) paper dealing with random parallel methods for nonconvex F ’s is the arxiv submission [38], which however does not enjoy the key properties 1, ...