Citation Context

... reducing the complexity of each iteration to the cost of an SVD [6]. However, for our low-rank and sparse decomposition problem, this form of iterative thresholding converges slowly, requiring up to 104 iterations. Ma et. al. [20, 37] suggest improving convergence using continuation techniques, and also demonstrate how Bregman iterations [42] can be applied to nuclear norm minimization. The convergence of iterative thresholding has also been greatly improved using ideas from Nesterov’s optimal first-order algorithm for smooth minimization [38], which was extended to nonsmooth optimization in [2, 39], and applied to `1-minimization in [2, 3, 40]. Based on [2], Toh et. al. developed a proximal gradient algorithm for matrix completion which they termed Accelerated Proximal Gradient (APG). A very similar APG algorithm was suggested for low-rank and sparse decomposition in [34]. That algorithm inherits the optimal O(1/k2) convergence rate for this class of problems. Empirical evidence suggests that these algorithms can solve the convex PCP problem at least 50 times faster than straightforward iterative thresholding (for more details and comparisons, see [34]). However, despite its good conver...

Citation Context

...algorithm in a more general framework and established connections to other known algorithms. In this paper, we prove that the proposed algorithm converges with rate O(1/N) for the primal-dual gap. In =-=[25]-=-, Nesterov showed that this rate of convergence is optimal for the class of convex optimization problems with known structure. Hence, our primal-dual algorithm is optimal in some sense. We further sho...

Citation Context

...convex with respect to the norm ‖·‖ ψt , the function ψ∗ t has η-Lipschitz continuous gradients with respect to ‖·‖ ψ ∗ t : ‖∇ψ ∗ t(g1)−∇ψ ∗ t(g2)‖ ψt ≤ η‖g1− g2‖ ψ ∗ t (30) for any g1,g2 (see, e.g., =-=Nesterov, 2005-=-, Theorem 1 or Hiriart-Urruty and Lemaréchal, 1996, Chapter X). Further, a simple argument with the fundamental theorem of calculus gives that if f has L-Lipschitz gradients, f(y)≤ f(x)+〈∇ f(x),y−x〉+(...

Citation Context

...particular problem can be expressed as a saddle-point problem which is well suited to recent algorithms based on a smoothing argument combined with an optimal firstorder smooth minimization algorithm =-=[21, 17, 2]-=-. These algorithms offer a significant reduction in computational time compared to generic interior-point SDP solvers. This also represents a change in the granularity of the solver, requiring a large...

Citation Context

...d promising numerical results for wavelet-based image deblurring. This algorithm is in the class of accelerated proximal gradient algorithms that were studied by Nesterov, Nemirovski, and others; see =-=[32, 33, 34, 36, 43]-=- and references therein. These accelerated proximal gradient algorithms have an attractive iteration complexity of O(1/ √ ɛ) for achieving ɛ-optimality; see Section 2. We extend Beck and Teboulle’s al...

Citation Context

... without scalable algorithms for solving the associated convex program. To this end, we discuss how a near-solution to this convex program can be obtained relatively efficiently via proximal gradient =-=[18, 19]-=- and iterative thresholding techniques, similar to those proposed for matrix completion in [20, 21]. For large matrices, these algorithms are significantly faster and more scalable than general-purpos...