Citation Context

... the ℓ0-minimization problem (1.1) min x∈R n ‖x‖0 subject to (s.t.) Ax = b, where ‖x‖0 := |{i, xi = 0}| and K ≤ m ≤ n (often K ≪ m ≪ n). Moreover, Candes, Romberg, and Tao (see [11, 12, 13]), Donoho =-=[21]-=-, and their colleagues have shown that, under some reasonable conditions on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. ...

Citation Context

...n A ∈ R m×n by solving the ℓ0-minimization problem (1.1) min x∈R n ‖x‖0 subject to (s.t.) Ax = b, where ‖x‖0 := |{i, xi = 0}| and K ≤ m ≤ n (often K ≪ m ≪ n). Moreover, Candes, Romberg, and Tao (see =-=[11, 12, 13]-=-), Donoho [21], and their colleagues have shown that, under some reasonable conditions on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈...

Citation Context

...e solution of problem (1.1). Greedy algorithms work when the data satisfy certain conditions, such as the restricted isometry property [13]. These algorithms include Orthogonal Matching Pursuit (OMP) =-=[42, 53]-=-, Stagewise OMP (StOMP) [23], CoSaMP [45], Subspace Pursuit (SP) [17], and many other variants. These algorithms, by and large, involve solving a sequence of subspace optimization problems of the form...

Citation Context

...n A ∈ R m×n by solving the ℓ0-minimization problem (1.1) min x∈R n ‖x‖0 subject to (s.t.) Ax = b, where ‖x‖0 := |{i, xi = 0}| and K ≤ m ≤ n (often K ≪ m ≪ n). Moreover, Candes, Romberg, and Tao (see =-=[11, 12, 13]-=-), Donoho [21], and their colleagues have shown that, under some reasonable conditions on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈...

Citation Context

...e solution of problem (1.1). Greedy algorithms work when the data satisfy certain conditions, such as the restricted isometry property [13]. These algorithms include Orthogonal Matching Pursuit (OMP) =-=[42, 53]-=-, Stagewise OMP (StOMP) [23], CoSaMP [45], Subspace Pursuit (SP) [17], and many other variants. These algorithms, by and large, involve solving a sequence of subspace optimization problems of the form...

Citation Context

...s work when the data satisfy certain conditions, such as the restricted isometry property [13]. These algorithms include Orthogonal Matching Pursuit (OMP) [42, 53], Stagewise OMP (StOMP) [23], CoSaMP =-=[45]-=-, Subspace Pursuit (SP) [17], and many other variants. These algorithms, by and large, involve solving a sequence of subspace optimization problems of the form (1.3) min x ‖Ax − b‖ 2 2 s.t. xi =0∀i ∈...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

...algorithm (SPA) [1] that solves problem (1.4) with the quadratic penalty replaced by1834 Z. WEN, W. YIN, D. GOLDFARB, AND Y. ZHANG an “exact” ℓ2-penalty, the spectral gradient projection method GPSR =-=[29]-=-, the fixedpoint continuation method FPC [33] for the ℓ1-regularized problem (1.4), and the gradient method in [46] for minimizing the more general function J(x)+H(x), where J is nonsmooth, H is smoot...

Citation Context

...=sgn(y) ⊙ max {|y|−ν, 0} , yields the unique minimizer of the function ν‖x‖1 + 1 2 ‖x−y‖2 2. The iteration (2.1) has been independently proposed by different groups of researchers in various contexts =-=[4, 15, 19, 24, 25, 28, 33, 48]-=-. Various modifications and enhancements have been applied to (2.1), which has also been generalized to certain other nonsmooth functions; see [26, 5, 29, 63]. Although (2.1) is very easy to compute, ...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

... or linear constraints in [44, 6, 43, 7, 32], and LP and QP subproblems have been used to solve general nonlinear programs in [8, 9]. The difference between our algorithm and the active-set algorithm =-=[40]-=- lies in the way in which the working index set is chosen. Thanks to the solution sparsity, our approach is more aggressive and effective. Our algorithm is different from any existing method in the li...

Citation Context

...RB, AND Y. ZHANG an “exact” ℓ2-penalty, the spectral gradient projection method GPSR [29], the fixedpoint continuation method FPC [33] for the ℓ1-regularized problem (1.4), and the gradient method in =-=[46]-=- for minimizing the more general function J(x)+H(x), where J is nonsmooth, H is smooth, and both are convex. In this paper, we propose an alternating-stage algorithm that combines the good features of...

Citation Context

...f the matrix-vector products spent on various tasks in the shrinkage and subspace optimization stages are also presented. We present our numerical results by using performance profiles as proposed in =-=[20]-=-. These profiles provide a way to graphically compare the quantities tp,s, suchas the number of iterations or CPU time required to solve problem p by each solver s. Define rp,s to be the ratio between...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

...−b‖ denotes the l2-norm of the residual, and nMat denotes the total number matrix-vector products involving A or A⊤ . We also use “nnzx” to denote the number of nonzeros in x which we estimate (as in =-=[5]-=-) by the minimum cardinality of a subset of the components of x that account for 99.5% of ‖x‖1, i.e., { (7.1) nnzx := min |Ω| : ∑ } { |xi| > 0.995‖x‖1 = min k : i∈Ω k∑ } |x(i)| ≥ 0.995‖x‖1 where x (i)...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

...mk→∞‖d k ‖ = 0. It is also proved that the sequence {x k } is at least R-linearly convergent under some mild assumptions. We now specify a strategy, which is based on the Barzilai-Borwein method (BB) =-=[2]-=-, for choosing the parameter λ k . The shrinkage iteration (2.2) first takes a gradient descent step with step size λ k along the negative gradient direction gk of the smooth function f(x), and then a...

Citation Context

...rithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms [28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49], the interior-point algorithm ℓ1 ℓs =-=[36]-=-, SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic penalty replaced by1834 Z. WEN, W. YIN,...

Citation Context

...edy algorithms work when the data satisfy certain conditions, such as the restricted isometry property [13]. These algorithms include Orthogonal Matching Pursuit (OMP) [42, 53], Stagewise OMP (StOMP) =-=[23]-=-, CoSaMP [45], Subspace Pursuit (SP) [17], and many other variants. These algorithms, by and large, involve solving a sequence of subspace optimization problems of the form (1.3) min x ‖Ax − b‖ 2 2 s....

Citation Context

...ion of the previous reference value Ck−1 and the function value ψμ(x k ), and as the iterations proceed, the weight on Ck is increased. For further information on nonmonotone line search methods, see =-=[18, 31, 52]-=-. Algorithm 2. Nonmonotone line search algorithm (NMLS). Initialization: Chooseastartingguessx 0 and parameters 0 <η<1, 0 <ρ<1, and 0 <λm <λM < ∞. SetC0 = ψμ(x 0 ), Q0 =1,andk =0. while “not converge”...

Citation Context

...chers in various contexts [4, 15, 19, 24, 25, 28, 33, 48]. Various modifications and enhancements have been applied to (2.1), which has also been generalized to certain other nonsmooth functions; see =-=[26, 5, 29, 63]-=-. Although (2.1) is very easy to compute, it can take more than thousands of iterations to achieve an acceptable accuracy for difficult problems. It is proved in [33] that (2.1) yields xk with the sam...

Citation Context

...n A ∈ R m×n by solving the ℓ0-minimization problem (1.1) min x∈R n ‖x‖0 subject to (s.t.) Ax = b, where ‖x‖0 := |{i, xi = 0}| and K ≤ m ≤ n (often K ≪ m ≪ n). Moreover, Candes, Romberg, and Tao (see =-=[11, 12, 13]-=-), Donoho [21], and their colleagues have shown that, under some reasonable conditions on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈...

Citation Context

...arch area. Examples of such algorithms include shrinkage-based algorithms [28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49], the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA =-=[3]-=- for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic penalty replaced by1834 Z. WEN, W. YIN, D. GOLDFARB, AND Y. ZHANG an “exact” ℓ2-pena...

Citation Context

...)) ⊤ (x −S(x − λg, μλ)) + μλ(‖x‖1 −‖S(x − λg, μλ)‖1) ≥ 0, which gives (3.3) g ⊤ d + μ(‖x + ‖1 −‖x‖1) ≤− 1 λ ‖d‖2 2 after rearranging terms. An alternative derivation of (3.3) is given in Lemma 2.1 in =-=[57]-=-. We can also reformulate (1.4) as (3.4) min f(x)+μξ s.t. (x, ξ) ∈ Ω, whose first-order optimality conditions for a stationary point x ∗ are (3.5) ∇f(x ∗ )(x − x ∗ )+μ(ξ −‖x ∗ ‖1) ≥ 0 ∀(x, ξ) ∈ Ω,183...

Citation Context

... . Therefore, our subspace optimization problem is (3.16) min ϕμ(x) s.t. x ∈ Ω(x k ), which can be solved by a limited-memory quasi-Newton method for problems with simple bound constraints (L-BFGS-B) =-=[67]-=-. In our implementations, we also consider subspace optimization without the bound constraints, i.e., (3.17) min x∈R n ϕμ(x) s.t. xi =0 ∀i ∈A(x k ). This is essentially an unconstrained minimization p...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

...tter able to take advantage of warm starts [47, 50]. For example, gradient projection and conjugate gradient steps have been combined to solve problems with bound constraints or linear constraints in =-=[44, 6, 43, 7, 32]-=-, and LP and QP subproblems have been used to solve general nonlinear programs in [8, 9]. The difference between our algorithm and the active-set algorithm [40] lies in the way in which the working in...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

... certain conditions, such as the restricted isometry property [13]. These algorithms include Orthogonal Matching Pursuit (OMP) [42, 53], Stagewise OMP (StOMP) [23], CoSaMP [45], Subspace Pursuit (SP) =-=[17]-=-, and many other variants. These algorithms, by and large, involve solving a sequence of subspace optimization problems of the form (1.3) min x ‖Ax − b‖ 2 2 s.t. xi =0∀i ∈ T, where T is a subset of t...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

...problem (1.4) min x∈Rn ψμ(x) :=μ‖x‖1+ 1 2 ‖Ax − b‖22, where μ>0. The theory for penalty functions implies that the solution (1.4) goes to the solution of (1.2) as μ goes to zero. It has been shown in =-=[65]-=- that (1.2) is equivalent to (1.4) for a suitable choice of b (which is different from the b in (1.4)). Furthermore, if the measurements are contaminated with noise, problem (1.4) is preferred. Other ...

Citation Context

...rithm FPC AS (version 1.1), we tested it on three different sets of problems and compared it with the state-of-the-art codes FPC (version 2.0) [34], spg bp in the software package SPGL1 (version 1.5) =-=[4]-=-, and CPLEX [36] with a MATLAB interface [1]. In subsection 7.1, we compare FPC AS to FPC, spg bp, and CPLEX for solving the basis pursuit problem (1.2) on a set of “pathological” problems with large ...

Citation Context

...her active set methods that follow this framework include those in which gradient projection and conjugate gradient methods are combined to solve problems with bound constraints or linear constraints =-=[44, 7, 43, 8, 31]-=- and those that combine linear and quadratic programming approaches to solve general nonlinear programming problems [9, 10]. In our approach, shrinkage is used to identify an active set in the first s...

Citation Context

...chers in various contexts [4, 15, 19, 24, 25, 28, 33, 48]. Various modifications and enhancements have been applied to (2.1), which has also been generalized to certain other nonsmooth functions; see =-=[26, 5, 29, 63]-=-. Although (2.1) is very easy to compute, it can take more than thousands of iterations to achieve an acceptable accuracy for difficult problems. It is proved in [33] that (2.1) yields xk with the sam...

Citation Context

...(|g k |−μ)‖2 provides a measure of the violation of the complementary conditions at the point xk . To calculate ξk, we use an identification function (3.24) ρ(x k √ ,ξk) := χ(xk ,μ,ξk)+ζk proposed in =-=[27]-=- for nonlinear programming that is based on the amount that the current iterate x k violates the optimality conditions for (1.4). Specifically, we set the threshold ξk initially to ξ0 = ¯ ξm andthenup...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

...gnals. Our alternating two-stage algorithm behaves like an active-set method. Compared with interior-point methods, active-set methods are more robust and better able to take advantage of warm starts =-=[47, 50]-=-. For example, gradient projection and conjugate gradient steps have been combined to solve problems with bound constraints or linear constraints in [44, 6, 43, 7, 32], and LP and QP subproblems have ...

Citation Context

...s on ¯x and A, the solution ¯x of problem (1.1) can be found by solving the basis pursuit (BP) problem (1.2) min x∈R n ‖x‖1 s.t. Ax = b. For more information on compressive sensing, see, for example, =-=[21, 54, 60, 61, 51, 41, 55, 37, 39, 38, 64]-=-. ∗Received by the editors January 24, 2009; accepted for publication (in revised form) March 29, 2010; published electronically June 23, 2010. http://www.siam.org/journals/sisc/32-4/74769.html † Depa...

Citation Context

...e features make the development of efficient optimization algorithms for compressive sensing applications an interesting research area. Examples of such algorithms include, shrinkage-based algorithms =-=[28, 48, 19, 3, 23, 24, 15, 34, 33, 63, 58, 49]-=-, the interior-point algorithm ℓ1 ℓs [37], SPGL1 [59] for the LASSO problem, the spectral gradient projection method GPSR [29] and the fixed-point continuation method FPC [34] for the ℓ1-regularized p...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

...tter able to take advantage of warm starts [47, 50]. For example, gradient projection and conjugate gradient steps have been combined to solve problems with bound constraints or linear constraints in =-=[44, 6, 43, 7, 32]-=-, and LP and QP subproblems have been used to solve general nonlinear programs in [8, 9]. The difference between our algorithm and the active-set algorithm [40] lies in the way in which the working in...

Citation Context

...gradient steps have been combined to solve problems with bound constraints or linear constraints in [44, 6, 43, 7, 32], and LP and QP subproblems have been used to solve general nonlinear programs in =-=[8, 9]-=-. The difference between our algorithm and the active-set algorithm [40] lies in the way in which the working index set is chosen. Thanks to the solution sparsity, our approach is more aggressive and ...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

...hat satisfies (3.7) with C k = ψμ(x k ). Such a line search method is monotone since ψμ(x k+1 ) <ψμ(x k ). Instead of using it, we use a nonmonotone line search method based on a strategy proposed in =-=[66]-=- (See algorithm NMLS (Algorithm 2)). In this method, the reference value Ck in the Armijo-like condition (3.7) is taken as a convex combination of the previous reference value Ck−1 and the function va...

Citation Context

...gradient steps have been combined to solve problems with bound constraints or linear constraints in [44, 6, 43, 7, 32], and LP and QP subproblems have been used to solve general nonlinear programs in =-=[8, 9]-=-. The difference between our algorithm and the active-set algorithm [40] lies in the way in which the working index set is chosen. Thanks to the solution sparsity, our approach is more aggressive and ...

Citation Context

...tter able to take advantage of warm starts [47, 50]. For example, gradient projection and conjugate gradient steps have been combined to solve problems with bound constraints or linear constraints in =-=[44, 6, 43, 7, 32]-=-, and LP and QP subproblems have been used to solve general nonlinear programs in [8, 9]. The difference between our algorithm and the active-set algorithm [40] lies in the way in which the working in...

Citation Context

...tter able to take advantage of warm starts [47, 50]. For example, gradient projection and conjugate gradient steps have been combined to solve problems with bound constraints or linear constraints in =-=[44, 6, 43, 7, 32]-=-, and LP and QP subproblems have been used to solve general nonlinear programs in [8, 9]. The difference between our algorithm and the active-set algorithm [40] lies in the way in which the working in...

Citation Context

...hological problems described in Table 3. Only the performance of FPC AS CG is reported because FPC AS BD performed similarly. The first test set includes four problems, CaltechTest1, ...,CaltechTest4 =-=[10]-=-, which are pathological because the magnitudes of the nonzero entries of the exact solutions ¯x lie in a large range. Such pathological problems are exaggerations of a large number of realistic probl...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

...ion of the previous reference value Ck−1 and the function value ψμ(x k ), and as the iterations proceed, the weight on Ck is increased. For further information on nonmonotone line search methods, see =-=[18, 31, 52]-=-. Algorithm 2. Nonmonotone line search algorithm (NMLS). Initialization: Chooseastartingguessx 0 and parameters 0 <η<1, 0 <ρ<1, and 0 <λm <λM < ∞. SetC0 = ψμ(x 0 ), Q0 =1,andk =0. while “not converge”...

Citation Context

...ion of the previous reference value Ck−1 and the function value ψμ(x k ), and as the iterations proceed, the weight on Ck is increased. For further information on nonmonotone line search methods, see =-=[18, 31, 52]-=-. Algorithm 2. Nonmonotone line search algorithm (NMLS). Initialization: Chooseastartingguessx 0 and parameters 0 <η<1, 0 <ρ<1, and 0 <λm <λM < ∞. SetC0 = ψμ(x 0 ), Q0 =1,andk =0. while “not converge”...

Citation Context

...order method. These features make the development of efficient optimization algorithms for CS applications an interesting research area. Examples of such algorithms include shrinkage-based algorithms =-=[28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49]-=-, the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) [1] that solves problem (1.4) with the quadratic...

Citation Context

... reduced Hessian of f(x) restricted to the support of x∗ . Various modifications and enhancements have been applied to (2.2), which has also been generalized to certain other nonsmooth functions; see =-=[25, 6, 29, 63]-=-. The first stage of our approach is based on the iterative shrinkage scheme (2.2). Once a “good” approximate solution xk of the ℓ1-regularized problem (1.4) is obtained from (2.2), the set of indices...

Citation Context

... Algorithm NMLS requires only that λk be bounded, and other strategies could easily be adopted. 3.1.1. An exact line search. An exact line search is possible if ψμ(·) isa piecewise quadratic function =-=[16, 30]-=-. We want to solve (3.12) min α∈[0,1] ψμ(x + αd) :=μ‖x+ αd‖1 + 1 2 ‖A(x + αd) − b‖22 = μ‖x + αd‖1 + 1 2 c1α 2 + c2α + c3, where c1 = ‖Ad‖2 2 , c2 =(Ad) ⊤ (Ax − b), and c3 =0.5 ‖Ax − b‖2 2 . The break ...

Citation Context

... Algorithm NMLS requires only that λk be bounded, and other strategies could easily be adopted. 3.1.1. An exact line search. An exact line search is possible if ψμ(·) isa piecewise quadratic function =-=[16, 30]-=-. We want to solve (3.12) min α∈[0,1] ψμ(x + αd) :=μ‖x+ αd‖1 + 1 2 ‖A(x + αd) − b‖22 = μ‖x + αd‖1 + 1 2 c1α 2 + c2α + c3, where c1 = ‖Ad‖2 2 , c2 =(Ad) ⊤ (Ax − b), and c3 =0.5 ‖Ax − b‖2 2 . The break ...

Citation Context

...ke (3.11) λ k =max{λm, min{λ k,BB1 ,λM }} or λ k =max{λm, min{λ k,BB2 ,λM }}, where 0 <λm ≤ λM < ∞ are fixed parameters. We should point out that the idea of using BB steps in CS has also appeared in =-=[29, 34, 63]-=-. However, Algorithm NMLS requires only that λk be bounded, and other strategies could easily be adopted. 3.1.1. An exact line search. An exact line search is possible if ψμ(·) isa piecewise quadratic...

Citation Context

... 3(f). On this measure, FPC AS CG and FPC AS BD performedbetterthantheSPGL1 solvers. 4.3. Sparco collection. In this subsection, we compare FPC AS and spg bp on 13 problems from the Sparco collection =-=[59]-=- (also see [58]). The parameters of spg bp and spg bp sub were set to their default values, and the parameters μ =10−10, ɛ =10−12,andɛx=10−16 were set for the two variants of FPC AS. A summary of the ...

Citation Context

...that λ k be bounded and other strategies could easily be adopted. 3.1. An exact line search. An exact line search is possible if ψµ(·) is a piecewise quadratic function. We want to solve (3.12) min α∈=-=[0,1]-=- ψµ(x + αd) := µ‖x + αd‖1 + 1 2 ‖A(x + αd) − b‖22, = µ‖x + αd‖1 + 1 2 c1α 2 + c2α + c3, where c1 = ‖Ad‖ 2 2, c2 = (Ad) ⊤ (Ax − b) and c3 = ‖Ax − b‖ 2 2. The break points of ψµ(x + αd) are {αi = −xi/di...

Citation Context

...thms [28, 48, 19, 4, 24, 25, 14, 33, 63, 56, 49], the interior-point algorithm ℓ1 ℓs [36], SPGL1 [58] for the LASSO problem, NESTA [3] for the BP denoising problem, a smoothed penalty algorithm (SPA) =-=[1]-=- that solves problem (1.4) with the quadratic penalty replaced by1834 Z. WEN, W. YIN, D. GOLDFARB, AND Y. ZHANG an “exact” ℓ2-penalty, the spectral gradient projection method GPSR [29], the fixedpoin...

Citation Context

... theoretical properties of our algorithm, including global convergence, R-linear convergence, and the identification of the active set after a finite number of steps, are studied in a companion paper =-=[62]-=-. Our line search scheme is based on properties of the search direction determined by shrinkage (2.1)–(2.2). Let d k := d (λk ) (x k ) denote this direction, i.e., (3.1) d (λ) (x) :=x + − x, x + = S(x...