#### DMCA

## Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems (2007)

### Cached

### Download Links

Venue: | IEEE Journal of Selected Topics in Signal Processing |

Citations: | 520 - 16 self |

### Citations

5226 | Convex Analysis - Rockafellar - 1970 |

3998 | Regression shrinkage and selection via the lasso,”
- Tibshirani
- 1996
(Show Context)
Citation Context ...a posteriori criterion for estimating x from observations y = Ax + n, (2) where n is white Gaussian noise of variance σ 2 , and the prior on x is Laplacian (that is, log p(x) = −λ�x�1 + K) [1], [25], =-=[54]-=-. Problem (1) can also be viewed as a regularization M. Figueiredo is with the Instituto de Telecomunicações and Department of Electrical and Computer Engineering, Instituto Superior Técnico, 1049001 ... |

3543 | Compressed sensing
- Donoho
- 2006
(Show Context)
Citation Context ...n (1). In 2D, however, the techniques of this paper cannot be applied directly. Another intriguing new application for the optimization problems above is compressed sensing 1 (CS) [6], [7], [8], [9], =-=[18]-=-. Recent results show that a relatively small number of random projections of a sparse signal can contain most of its salient information. It follows that if a signal is sparse or approximately sparse... |

3136 |
A wavelet tour of signal processing
- Mallat
- 1998
(Show Context)
Citation Context ...d, then W must be a d × n matrix. If W contains an orthogonal wavelet basis (d = n), matrixvector products involving W or W T can be implemented using fast wavelet transform algorithms with O(n) cost =-=[40]-=-, instead of the O(n 2 ) cost of a direct matrix-vector product. Thus, the cost of a product by A or A T is O(n) plus that of multiplying by R or R T which, with a direct implementation, is O(k n). Wh... |

2683 | Atomic decomposition by basis pursuit
- Chen, Donoho, et al.
- 1998
(Show Context)
Citation Context ...o infer x from noiseless observations y = Ax or from noisy observations as in (2). The presence of the ℓ1 term encourages small components of x to become exactly zero, thus promoting sparse solutions =-=[11]-=-, [54]. Because of this feature, (1) has been used for more than three decades in several signal processing problems where sparseness is sought; some early references are [12], [37], [50], [53]. In th... |

2559 | Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information
- Candès, Romberg, et al.
- 2006
(Show Context)
Citation Context ...lation (1). In 2D, however, the techniques of this paper cannot be applied directly. Another intriguing new application for the optimization problems above is compressed sensing 1 (CS) [6], [7], [8], =-=[9]-=-, [18]. Recent results show that a relatively small number of random projections of a sparse signal can contain most of its salient information. It follows that if a signal is sparse or approximately ... |

2236 | Nonlinear total variation based noise removal algorithms
- Rudin, Osher, et al.
- 1992
(Show Context)
Citation Context ...mage/signal [24], [25], [26].sTO APPEAR IN THE IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, 2007. 2 We mention also image restoration problems under total variation (TV) regularization [10], =-=[47]-=-. In the onedimensional (1D) case, a change of variables leads to the formulation (1). In 2D, however, the techniques of this paper cannot be applied directly. Another intriguing new application for t... |

1477 | Signal recovery from random projections
- Candés, JK
- 2005
(Show Context)
Citation Context ... the formulation (1). In 2D, however, the techniques of this paper cannot be applied directly. Another intriguing new application for the optimization problems above is compressed sensing 1 (CS) [6], =-=[7]-=-, [8], [9], [18]. Recent results show that a relatively small number of random projections of a sparse signal can contain most of its salient information. It follows that if a signal is sparse or appr... |

1361 | Stable signal recovery from incomplete and inaccurate measurements
- Candes, Romberg, et al.
- 2006
(Show Context)
Citation Context ...ds to the formulation (1). In 2D, however, the techniques of this paper cannot be applied directly. Another intriguing new application for the optimization problems above is compressed sensing 1 (CS) =-=[6]-=-, [7], [8], [9], [18]. Recent results show that a relatively small number of random projections of a sparse signal can contain most of its salient information. It follows that if a signal is sparse or... |

1292 | Least angle regression
- Efron, Hastie, et al.
- 2004
(Show Context)
Citation Context ...RW. Homotopy algorithms that find the full path of solutions, for all nonnegative values of the scalar parameters in the various formulations (τ in (1), ε in (3), and t in (4)), have been proposed in =-=[22]-=-, [39], [46], and [57]. The formulation (4) is addressed in [46], while [57] addresses (1) and (4). 1 A comprehensive repository of CS literature and software can be fond in www.dsp.ece.rice.edu/cs/. ... |

1234 | De-noising by soft-thresholding
- Donoho
- 1995
(Show Context)
Citation Context ... used in other ℓ1-based algorithms, e.g., [42]. It is also worth pointing out that debiasing is not always desirable. Shrinking the selected coefficients can mitigate unusually large noise deviations =-=[19]-=-, a desirable effect that may be undone by debiasing. F. Warm Starting and Continuation The gradient projection approach benefits from a good starting point. This suggests that we can use the solution... |

901 | Greed is good: algorithmic results for sparse approximation
- Tropp
- 2004
(Show Context)
Citation Context ...For this reason, IST may not be as effective for solving (1) in CS applications, as it is in deconvolution problems. Finally, we mention matching pursuit (MP) and orthogonal MP (OMP) [5], [17], [20], =-=[56]-=-, which are greedy schemes to find a sparse representation of a signal on a dictionary of functions. (Matrix A is seen as an n-element dictionary of k-dimensional signals). MP works by iteratively cho... |

856 | The Dantzig selector: statistical estimation when p is much larger than n
- Candès, Tao
- 2007
(Show Context)
Citation Context ...formulation (1). In 2D, however, the techniques of this paper cannot be applied directly. Another intriguing new application for the optimization problems above is compressed sensing 1 (CS) [6], [7], =-=[8]-=-, [9], [18]. Recent results show that a relatively small number of random projections of a sparse signal can contain most of its salient information. It follows that if a signal is sparse or approxima... |

755 | The Linear Complementarity Problem
- Cottle, Pang, et al.
- 1992
(Show Context)
Citation Context ...re is a constant CLCP such that dist(z, S) ≤ CLCP �min(z, ∇F(z))� where S denotes the solution set of (8), dist(·) is the distance operator, and the min on the right-hand side is taken component-wise =-=[14]-=-. With this bound in mind, we can define a convergence criterion as follows: �min(z, ∇F(z))� ≤ tolP. (17) A third criterion proposed recently in [36] is based on duality theory for the original formul... |

732 | An iterative thresholding algorithm for linear inverse problems with a sparsity constraint
- Daubechies, Defrise, et al.
- 2004
(Show Context)
Citation Context ... OF SELECTED TOPICS IN SIGNAL PROCESSING, 2007. 3 as an EM algorithm, in the context of image deconvolution problems [45], [25]. IST can also be derived in a majorizationminimization (MM) framework 2 =-=[16]-=-, [26] (see also [23], for a related algorithm derived from a different perspective). Convergence of IST algorithms was shown in [13], [16]. IST algorithms are based on bounding the matrix A T A (the ... |

644 | LSQR: An algorithm for sparse linear equations and sparse least squares
- Paige, Saunders
- 1982
(Show Context)
Citation Context ...he unknowns), then applying a standard primal-dual IP approach [60]. The linear equations or least-squares problems that arise at each IP iteration are then solved with iterative methods such as LSQR =-=[48]-=- or conjugate gradients (CG). Each iteration of these methods requires one multiplication each by A and A T . MATLAB implementations of related approaches are available in the SparseLab toolbox; see i... |

616 | An algorithm for total variation minimization and applications
- Chambolle
- 2004
(Show Context)
Citation Context ...nown image/signal [24], [25], [26].sTO APPEAR IN THE IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, 2007. 2 We mention also image restoration problems under total variation (TV) regularization =-=[10]-=-, [47]. In the onedimensional (1D) case, a change of variables leads to the formulation (1). In 2D, however, the techniques of this paper cannot be applied directly. Another intriguing new application... |

596 | Primal-Dual Interior Point Methods
- Wright
- 1997
(Show Context)
Citation Context ...lating them as “perturbed linear programs” (which are linear programs with additional terms in the objective which are squared norms of the unknowns), then applying a standard primal-dual IP approach =-=[60]-=-. The linear equations or least-squares problems that arise at each IP iteration are then solved with iterative methods such as LSQR [48] or conjugate gradients (CG). Each iteration of these methods r... |

520 | Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine
- Lustig, Donoho, et al.
(Show Context)
Citation Context ...s very small (say l pixels) these products can be done with even lower cost, O(kl), by implementing the corresponding convolution. Also, in certain applications of CS, such as MR image reconstruction =-=[38]-=-, R is formed from a subset of the discrete Fourier transform basis, so the cost is O(k log k) using the FFT. IV. EXPERIMENTS This section describes some experiments testifying to the very good perfor... |

498 | Signal recovery by proximal forward-backward splitting,”Multiscale Modeling
- Combettes, Wajs
- 2005
(Show Context)
Citation Context ...can also be derived in a majorizationminimization (MM) framework 2 [16], [26] (see also [23], for a related algorithm derived from a different perspective). Convergence of IST algorithms was shown in =-=[13]-=-, [16]. IST algorithms are based on bounding the matrix A T A (the Hessian of �y − Ax� 2 2 ) by a diagonal D (i.e., D − AT A is positive semi-definite), thus attacking (1) by solving a sequence of sim... |

478 | relax: convex programming methods for identifying sparse signals
- Tropp
- 2006
(Show Context)
Citation Context ... Section 5]) criterion and the least absolute shrinkage and selection operator (LASSO, [54]). For brief historical accounts on the use of the ℓ1 penalty in statistics and signal processing, see [41], =-=[55]-=-. Problem (1) is closely related to the following convex constrained optimization problems: and min x �x�1 subject to �y − Ax� 2 2 ≤ ε (3) min x �y − Ax� 2 2 subject to �x�1 ≤ t, (4) where ε and t are... |

453 | Stable recovery of sparse overcomplete representations in the presence of noise
- Donoho, Elad, et al.
(Show Context)
Citation Context ...umns. For this reason, IST may not be as effective for solving (1) in CS applications, as it is in deconvolution problems. Finally, we mention matching pursuit (MP) and orthogonal MP (OMP) [5], [17], =-=[20]-=-, [56], which are greedy schemes to find a sparse representation of a signal on a dictionary of functions. (Matrix A is seen as an n-element dictionary of k-dimensional signals). MP works by iterative... |

361 |
Subset selection in regression
- Miller
- 2002
(Show Context)
Citation Context ..., [11, Section 5]) criterion and the least absolute shrinkage and selection operator (LASSO, [54]). For brief historical accounts on the use of the ℓ1 penalty in statistics and signal processing, see =-=[41]-=-, [55]. Problem (1) is closely related to the following convex constrained optimization problems: and min x �x�1 subject to �y − Ax� 2 2 ≤ ε (3) min x �y − Ax� 2 2 subject to �x�1 ≤ t, (4) where ε and... |

347 | An EM algorithm for wavelet-based image restoration
- Figueiredo, Nowak
- 2003
(Show Context)
Citation Context ...ximum a posteriori criterion for estimating x from observations y = Ax + n, (2) where n is white Gaussian noise of variance σ 2 , and the prior on x is Laplacian (that is, log p(x) = −λ�x�1 + K) [1], =-=[25]-=-, [54]. Problem (1) can also be viewed as a regularization M. Figueiredo is with the Instituto de Telecomunicações and Department of Electrical and Computer Engineering, Instituto Superior Técnico, 10... |

304 |
Two point step size gradient methods
- Barzilai, Borwein
- 1988
(Show Context)
Citation Context ...SR-BB Algorithm Algorithm GPSR-Basic ensures that the objective function F decreases at every iteration. Recently, considerable attention has been paid to an approach due to Barzilai and Borwein (BB) =-=[2]-=- that does not have this property. This approach was originally developed in the context of unconstrained minimization of a smooth nonlinear function F . It calculates each step by the formula δ (k) =... |

299 | An algorithm for quadratic programming, - Frank, Wolfe - 1956 |

240 | A new approach to variable selection in least squares problems
- Osborne, Presnell, et al.
(Show Context)
Citation Context ... algorithms that find the full path of solutions, for all nonnegative values of the scalar parameters in the various formulations (τ in (1), ε in (3), and t in (4)), have been proposed in [22], [39], =-=[46]-=-, and [57]. The formulation (4) is addressed in [46], while [57] addresses (1) and (4). 1 A comprehensive repository of CS literature and software can be fond in www.dsp.ece.rice.edu/cs/. The method i... |

239 | Signal reconstruction fromnoisy randomprojections
- Haupt, NowakR
(Show Context)
Citation Context ...tion (e.g., a wavelet basis). Problem (1) is a robust version of this reconstruction process, which is resilient to errors and noisy data, and similar criteria have been proposed and analyzed in [8], =-=[32]-=-. B. Previous Algorithms Several optimization algorithms and codes have been proposed to solve the QCLP (3), the QP (4), the LP (5), and the unconstrained (but nonsmooth) formulation (1). We review th... |

208 | M.: Nonmonotone spectral projected gradient methods on convex sets
- Birgin, Martińez, et al.
- 2000
(Show Context)
Citation Context .... C. Convergence Convergence of the methods proposed above can be derived from the analysis of Bertsekas [3] and Iusem [34], but follows most directly from the results of Birgin, Martinez, and Raydan =-=[4]-=- and Serafini, Zanghirati, and Zanni [52]. We summarize convergence properties of the two algorithms described above, assuming that termination occurs only when z (k+1) = z (k) (which indicates that z... |

118 | Why simple shrinkage is still relevant for redundant representations
- Elad
- 2005
(Show Context)
Citation Context ...N SIGNAL PROCESSING, 2007. 3 as an EM algorithm, in the context of image deconvolution problems [45], [25]. IST can also be derived in a majorizationminimization (MM) framework 2 [16], [26] (see also =-=[23]-=-, for a related algorithm derived from a different perspective). Convergence of IST algorithms was shown in [13], [16]. IST algorithms are based on bounding the matrix A T A (the Hessian of �y − Ax� 2... |

113 |
Simultaneous variable selection
- Turlach, Venables, et al.
- 2004
(Show Context)
Citation Context ...ignificant, and since these methods require at least as many pivot operations as there are nonzeros in the solution, they may be less competitive on such problems. The interior-point (IP) approach in =-=[58]-=-, which solves a generalization of (4), also requires explicit construction of A T A, though the approach could in principle modified to allow iterative solution of the linear system at each primal-du... |

102 |
On the solution of large quadratic programming problems with bound constraints
- Moré, Toraldo
- 1991
(Show Context)
Citation Context ...he difference in performance were very small, so we focus our presentation on the method described above. In earlier testing, we experimented with other variants of GP, including the GPCG approach of =-=[43]-=- and the proximal-point approach of [59]. The GPCG approach runs into difficulties because the projection of the Hessian B onto most faces of the positive orthant defined by z ≥ 0 is singular, so the ... |

91 |
Mol: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint
- Daubechies, Defrise, et al.
- 2004
(Show Context)
Citation Context ...ions involving and . Initially, IST was presented as an EM algorithm, in the context of image deconvolution problems [25], [45]. IST can also be derived in a majorization-minimization (MM) framework2 =-=[16]-=-, [26] (see also [23], for a related algorithm derived from a different perspective). Convergence of IST algorithms was shown in [13], [16]. IST algorithms are based on bounding the matrix (the Hessia... |

82 |
Homotopy continuation for sparse signal representation
- Malioutov, Cetin, et al.
- 2005
(Show Context)
Citation Context ...motopy algorithms that find the full path of solutions, for all nonnegative values of the scalar parameters in the various formulations (τ in (1), ε in (3), and t in (4)), have been proposed in [22], =-=[39]-=-, [46], and [57]. The formulation (4) is addressed in [46], while [57] addresses (1) and (4). 1 A comprehensive repository of CS literature and software can be fond in www.dsp.ece.rice.edu/cs/. The me... |

77 |
Greedy adaptive approximation
- Davis, Mallat, et al.
- 1997
(Show Context)
Citation Context ...an columns. For this reason, IST may not be as effective for solving (1) in CS applications, as it is in deconvolution problems. Finally, we mention matching pursuit (MP) and orthogonal MP (OMP) [5], =-=[17]-=-, [20], [56], which are greedy schemes to find a sparse representation of a signal on a dictionary of functions. (Matrix A is seen as an n-element dictionary of k-dimensional signals). MP works by ite... |

77 | Spectral bounds for sparse PCA: Exact and greedy algorithms
- Avidan
- 2006
(Show Context)
Citation Context ...ep chooses the optimal values for these components according to a least-squares criterion (without the regularization term τ�x�1). Similar techiques have been used in other ℓ1-based algorithms, e.g., =-=[42]-=-. It is also worth pointing out that debiasing is not always desirable. Shrinking the selected coefficients can mitigate unusually large noise deviations [19], a desirable effect that may be undone by... |

73 |
A bound optimization approach to wavelet-based image deconvolution
- Figueiredo, Nowak
(Show Context)
Citation Context ...ess that the goal of these experiments is not to assess the performance (e.g., in terms of SNR improvement) of the criterion form (1). Such an assessment has been comprehensively carried out in [25], =-=[26]-=-, and several other recent works on this topic. Rather, our goal is to compare the speed of the proposed GPSR algorithms against the competing IST. We consider three standard benchmark problems summar... |

70 |
Fast solution of l1-norm minimization problems when the solution may be sparse
- Donoho, Tsaig
- 2006
(Show Context)
Citation Context ...ate-of-the-art approaches, namely IST [16], [25], and the recent l1_ls package, which was shown in [36] to outperform all previous methods, including the ℓ1-magic toolbox and the homotopy method from =-=[21]-=-. The algorithms discussed in Section III are written in MATLAB and are freely available for download from www.lx.it.pt/˜mtf/GPSR/. For the GPSR-BB algorithm, we set αmin = 10 −30 , αmax = 10 30 ; the... |

54 | Implementation of warm-start strategies in interiorpoint methods for linear programming in fixed dimension
- John, Yıldırım
(Show Context)
Citation Context ...ods such as those in [11], [36], and ℓ1-magic have been less successful in making effective use of warmstart information, though this issue has been investigated in various contexts (see, e.g., [30], =-=[35]-=-, [61]). To benefit from a warm start, IP methods require the initial point to be not only close to the solution but also sufficiently interior to the feasible set and close to a “central path,” which... |

53 |
R.: Projected barzilai-borwein methods for large-scale box-constrained quadratic programming
- Dai, Fletcher
(Show Context)
Citation Context ...be effective on simple problems. Numerous variants have been proposed recently, and subjected to a good deal of theoretical and computational evaluation. The BB approach has been extended to BCQPs in =-=[15]-=-, [52]. The approach described here is simply that of [52, Section 2.2]. We choose λk in (12) as the exact minimizer over the interval [0, 1] and choose η (k) at each iteration in the manner described... |

49 |
A method for large-scale ℓ1-regularized least squares
- Kim, Koh, et al.
- 2007
(Show Context)
Citation Context ...olve a quadratic programming reformulation of (1), different from the one used here. Each search step is computed using preconditioned conjugate gradient (PCG) and requires only products by A and A T =-=[36]-=-. The code, available at www.stanford.edu/˜boyd/l1_ls/, is reported to be faster than competing codes on the problems tested in [36]. The ℓ1-magic suite of codes (which is available at www.l1-magic.or... |

41 |
A fixed-point continuation method for ℓ1-regularized minimization with applications to compressed sensing
- Hale, Yin, et al.
- 2007
(Show Context)
Citation Context ... LARS and other homotopy schemes, which compute solutions for a range of parameter values in succession. In particular, “warm-starting” allows using GPSR within a continuation scheme (as suggested in =-=[31]-=-). IP methods such as those in [11], [36], and ℓ1-magic have been less successful in making effective use of warmstart information, though this issue has been investigated in various contexts (see, e.... |

40 | Image denoising with shrinkage and redundant representations
- Elad, Matalon, et al.
- 2006
(Show Context)
Citation Context ...elet basis or a redundant dictionary (that is, multiplying by W corresponds to performing an inverse wavelet transform), and x is the vector of representation coefficients of the unknown image/signal =-=[24]-=-, [25], [26].sTO APPEAR IN THE IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, 2007. 2 We mention also image restoration problems under total variation (TV) regularization [10], [47]. In the one... |

39 | Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution - Levy, Fullagar - 1981 |

29 | Multipath time-delay detection and estimation
- Fuchs
- 1999
(Show Context)
Citation Context ...ath,” which is difficult to satisfy in practice. II. PROPOSED FORMULATION A. Formulation as a Quadratic Program The first key step of our GPSR approach is to express (1) as a quadratic program; as in =-=[28]-=-, this is done by splitting the variable x into its positive and negative parts. Formally, we introduce vectors u and v and make the substitution x = u − v, u ≥ 0, v ≥ 0. (6) These relationships are s... |

25 |
Deconvolution with the ℓ1 norm
- Taylor, Banks, et al.
- 1979
(Show Context)
Citation Context ...utions [11], [54]. Because of this feature, (1) has been used for more than three decades in several signal processing problems where sparseness is sought; some early references are [12], [37], [50], =-=[53]-=-. In the 1990’s, seminal work on the use of ℓ1 sparseness-inducing penalties/log-priors appeared in the literature: the now famous basis pursuit denoising (BPDN, [11, Section 5]) criterion and the lea... |

23 | Robust modeling of erratic data - Claerbout, Muir - 1973 |

23 | More on sparse representations in arbitrary bases
- Fuchs
- 2004
(Show Context)
Citation Context ...vation y is generated according to (2), with σ 2 = 10 −4 . Parameter τ is chosen as suggested in [36]: τ = 0.1 �A T y�∞; (22) notice that for τ ≥ �A T y�∞ the unique minimum of (1) is the zero vector =-=[29]-=-, [36]. The original signal and the estimate obtained by solving (1) using the monotone version of the GPSR-BB (which is essentially the same as that produced by the nonmonotone GPSR-BB and GPSR-Basic... |

22 |
Fast wavelet-based image deconvolution using the EM algorithm
- Nowak, Figueiredo
- 2001
(Show Context)
Citation Context ...ns involving A and A T . Initially, IST was presentedsTO APPEAR IN THE IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, 2007. 3 as an EM algorithm, in the context of image deconvolution problems =-=[45]-=-, [25]. IST can also be derived in a majorizationminimization (MM) framework 2 [16], [26] (see also [23], for a related algorithm derived from a different perspective). Convergence of IST algorithms w... |

16 | A new unblocking technique to warmstart interior point methods based on sensitivity analysis
- Gondzio, Grothey
(Show Context)
Citation Context ...P methods such as those in [11], [36], and ℓ1-magic have been less successful in making effective use of warmstart information, though this issue has been investigated in various contexts (see, e.g., =-=[30]-=-, [35], [61]). To benefit from a warm start, IP methods require the initial point to be not only close to the solution but also sufficiently interior to the feasible set and close to a “central path,”... |

15 |
On algorithms for solving least squares problems under an L1 penalty or an L1 constraint
- Turlach
- 2005
(Show Context)
Citation Context ...s that find the full path of solutions, for all nonnegative values of the scalar parameters in the various formulations (τ in (1), ε in (3), and t in (4)), have been proposed in [22], [39], [46], and =-=[57]-=-. The formulation (4) is addressed in [46], while [57] addresses (1) and (4). 1 A comprehensive repository of CS literature and software can be fond in www.dsp.ece.rice.edu/cs/. The method in [39] pro... |

14 |
Gradient projection methods for large quadratic programs and applications in training support vector machines
- Serafini, Zanghirati, et al.
(Show Context)
Citation Context ...ective on simple problems. Numerous variants have been proposed recently, and subjected to a good deal of theoretical and computational evaluation. The BB approach has been extended to BCQPs in [15], =-=[52]-=-. The approach described here is simply that of [52, Section 2.2]. We choose λk in (12) as the exact minimizer over the interval [0, 1] and choose η (k) at each iteration in the manner described above... |

13 |
Implementing proximal point methods for linear programming
- Wright
- 1990
(Show Context)
Citation Context ...mall, so we focus our presentation on the method described above. In earlier testing, we experimented with other variants of GP, including the GPCG approach of [43] and the proximal-point approach of =-=[59]-=-. The GPCG approach runs into difficulties because the projection of the Hessian B onto most faces of the positive orthant defined by z ≥ 0 is singular, so the inner CG loop in this algorithm tends to... |

13 | On the convergence properties of the projected gradient method for convex optimization - Iusem - 2003 |

8 |
An algorithm for the minimization of mixed ` and ` norms with application to Bayesian estimation
- Alliney, Ruzinsky
- 1994
(Show Context)
Citation Context ... a Bayesian perspective, (1) can be seen as a maximum a posteriori criterion for estimating from observations (2) where is white Gaussian noise of variance , and the prior on is Laplacian (that is, ) =-=[1]-=-, [25], [54]. Problem (1) can also be viewed as a regularization technique to overcome the ill-conditioned, or even singular, nature of matrix , when trying to infer from noiseless observations or fro... |

7 | Convex Analysis Princeton, N.J., Princeton University Press Ruppert, D. (2002): Selecting the Number of Knots for Penalized Splines - Rockafellar - 1970 |

6 |
PDCO: primal-dual interior-point method for convex objectives,” Systems Optimization
- Saunders
- 2002
(Show Context)
Citation Context ...one multiplication each by A and A T . MATLAB implementations of related approaches are available in the SparseLab toolbox; see in particular the routines SolveBP and pdco. For additional details see =-=[51]-=-. Another IP method was recently proposed to solve a quadratic programming reformulation of (1), different from the one used here. Each search step is computed using preconditioned conjugate gradient ... |

5 |
An algorithm for the minimization of mixed ℓ1 and ℓ2 norms with application to Bayesian estimation
- Alliney, Ruzinsky
- 1994
(Show Context)
Citation Context ... a maximum a posteriori criterion for estimating x from observations y = Ax + n, (2) where n is white Gaussian noise of variance σ 2 , and the prior on x is Laplacian (that is, log p(x) = −λ�x�1 + K) =-=[1]-=-, [25], [54]. Problem (1) can also be viewed as a regularization M. Figueiredo is with the Instituto de Telecomunicações and Department of Electrical and Computer Engineering, Instituto Superior Técni... |

5 |
A fixed-point continuation method for -regularized minimization with applications to compressed sensing
- Hale, Yin, et al.
- 2007
(Show Context)
Citation Context ... LARS and other homotopy schemes, which compute solutions for a range of parameter values in succession. In particular, “warm-starting” allows using GPSR within a continuation scheme (as suggested in =-=[31]-=-). IP methods such as those in [11], [36], and -magic have been less successful in making effective use of warm-start information, though this issue has been investigated in various contexts (see, e.g... |

5 |
Linear inversion of band-limited reflection histograms
- Santosa, Symes
- 1986
(Show Context)
Citation Context ...se solutions [11], [54]. Because of this feature, (1) has been used for more than three decades in several signal processing problems where sparseness is sought; some early references are [12], [37], =-=[50]-=-, and [53]. In the 1990s, seminal work on the use of sparseness-inducing penalties/log-priors appeared in the literature: the now famous basis pursuit denoising (BPDN, [11, Section 5]) criterion and t... |

4 |
Building-Cube Method for Large-Scale
- Nakahashi, Kim
(Show Context)
Citation Context ...d to solve a quadratic programming reformulation of (1), different from the one used here. Each search step is computed using preconditioned conjugate gradient (PCG) and requires only products by and =-=[36]-=-. The code, available at http://www.stanford.edu/~boyd/l1_ls/, is reported to be faster than competing codes on the problems tested in [36]. The -magic suite of codes (which is available at http://www... |

3 |
Deconvolution with the ` norm
- Taylor, Bank, et al.
- 1979
(Show Context)
Citation Context ...ns [11], [54]. Because of this feature, (1) has been used for more than three decades in several signal processing problems where sparseness is sought; some early references are [12], [37], [50], and =-=[53]-=-. In the 1990s, seminal work on the use of sparseness-inducing penalties/log-priors appeared in the literature: the now famous basis pursuit denoising (BPDN, [11, Section 5]) criterion and the least a... |

2 |
Linear invesion of band-limited reflection histograms
- Santosa, Symes
- 1986
(Show Context)
Citation Context ...se solutions [11], [54]. Because of this feature, (1) has been used for more than three decades in several signal processing problems where sparseness is sought; some early references are [12], [37], =-=[50]-=-, [53]. In the 1990’s, seminal work on the use of ℓ1 sparseness-inducing penalties/log-priors appeared in the literature: the now famous basis pursuit denoising (BPDN, [11, Section 5]) criterion and t... |

1 |
Gradient pursuits”, submitted
- Blumensath, Davies
- 2007
(Show Context)
Citation Context ...ws than columns. For this reason, IST may not be as effective for solving (1) in CS applications, as it is in deconvolution problems. Finally, we mention matching pursuit (MP) and orthogonal MP (OMP) =-=[5]-=-, [17], [20], [56], which are greedy schemes to find a sparse representation of a signal on a dictionary of functions. (Matrix A is seen as an n-element dictionary of k-dimensional signals). MP works ... |

1 |
On the convergence proeperties of the projected gradient method for convex optimization
- Iusem
- 2003
(Show Context)
Citation Context ...efined by z ≥ 0 is singular, so the inner CG loop in this algorithm tends to fail. C. Convergence Convergence of the methods proposed above can be derived from the analysis of Bertsekas [3] and Iusem =-=[34]-=-, but follows most directly from the results of Birgin, Martinez, and Raydan [4] and Serafini, Zanghirati, and Zanni [52]. We summarize convergence properties of the two algorithms described above, as... |