J. Fessler and A. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 1417--1429, 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... with convergence speed in iteration counts which is very similar to direct ICD [10] One may also solve the EM formulation pixel sequentially, preserving the provable convergence of EM while substantially improving its speed, as in the space alternating EM algorithm (SAGE) of Fessler and Hero [15]. Column action methods can easily be made to converge reliably to the unique global minimum of the ML or MAP functional. Here we present two improvements to current forms of ICD: 1) global convergence of the approximate greedy descent algorithm follows from the introduction of a new local ....

....can be efficiently implemented via table look ups. B. Global Convergence of ICD FS In order to prove the global convergence of this new ICD FS algorithm, we simply verify that it meets the assumptions and necessary conditions of the global convergence proof presented by Fessler and Hero in [15] for convergence of SAGE under positivity constraints. Since this proof requires continuity of the log likelihood on , we must assume that the background noise is greater than zero, i.e. in emission case (1) We discuss alternative methods for the case later in this section. a) b) c) d) ....

[Article contains additional citation context not shown here]

J. Fessler and A. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 1417--1429, Oct. 1995.

....of the EM algorithm based on standard numerical tools to speed up the convergence. There are often effective, but they do not guarantee monotone increase in the objective function. To overcome this problem, alternatives based on model reduction ( 22] 14] and efficient data augmentation ( 6] [7], 9] 20] 21] 23] 15] see also the chapter 5 of [18] have recently been considered. These extensions share the simplicity and stability with EM while speeding up the convergence. However, as far as we know, only two extensions ( 24] 16] were devoted to speeding up the convergence in ....

....3, we describe our component wise algorithm and show, in Section 4, that it can also be interpreted as a proximal point algorithm. Using this interpretation, convergence of CEM proved in Section 5. Illustrative numerical experiments comparing the behaviors of EM, a version of the SAGE algorithm [6, 7] and CEM are presented in Section 6. A discussion section ends the paper. An appendix carefully describes the SAGE method in the mixture context in order to provide detailed comparison with the proposed . 2 EM type algorithms for mixtures We consider a J component mixture in R d g(yj ) ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Trans. Image Processing, 4:1417--1429, 1995.

....Another interesting recent method is the one in [27] The methods of [13] 27] 25] 30] constitute the state of the art. Finally, we mention that EM and EM type algorithms have been previously used in image restoration and reconstruction, although not in a wavelet based formulation (e.g. [9], 10] 15] V. The Best of Both Worlds The approach proposed in this paper is able to use the best of the wavelet and Fourier worlds in image deconvolution problems. The speed and convenience of FFT based linear ltering, which is well matched to the observation model, and the adequacy of ....

J. Fessler, A. Hero. \Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. on Image Processing, vol. 4, pp. 1417-29, 1995.

....and Operations Research Department, George Mason University, Fairfax, VA 22030 4444 USA. Publisher Item Identifier S 0278 0062(00)02978 5. facts in later iterates of the expectation maximization (EM) algorithm [54] Regularization of the objective function is commonly referred to as penalized ML [10] or maximum a posteriori (MAP) reconstruction [35] and has generated significant interest because, in addition to improving the noise properties of the converged solution, inclusion of a regularization term in the objective function also leads to faster convergence [33] PET reconstructions of ....

.... of nonnegativity via a quadratic penalty term in the objective function [40] and in a separate paper explored active set methods [41] Coordinate ascent and grouped coordinate ascent methods have recently been reported to improve convergence significantly over various MAP EM algorithms [2] [10], 11] 20] While the results presented in these latter papers are quite strong, GaussSeidel iterations do not accommodate convenient system matrix factorizations such as those in [40] This paper introduces ML reconstruction methods that solve a sequence of subproblems that successively ....

J. A. Fessler, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 14101429, Oct., 1995.

.... uniquely existent and is termed as the accelerated cross reference maximum likelihood estimate (ACRMLE) New algorithms adapted from the expectation conditional maximization (ECM) algorithm in [14] as well as the space alternating generalized expectation maximization (SAGE) algorithms in [15] and [16] are developed in this article. These algorithms are shown to be convergent and have fast convergence speeds. Simulation studies reveal that an even faster convergence speed is achieved if one alternates the order of iterations. The penalty parameter can be selected by users or data driven ....

....by Theorem 2, Phi( new ; new d ) Phi( old ; old d ) According to Theorem 4, the modified ECM sequence ( t) t) d ) obtained from Algorithm 1 converges to the unique maximizer, ACRMLE1 ; d;ACRMLE1 ) C. The Modified SAGE Algorithm Fessler and Hero in [15] and [16] proposed another iterative methods to accelerate the convergence speed of the EM algorithm. They investigated a different approach in specifying the hidden data space such that the resulting space alternating generalized expectation maximization algorithm (SAGE) has a faster convergence speed ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized expectation-maximization algorithms," IEEE Trans. on Image Processing, vol. 4, no. 10, pp. 1417--1429, 1995.

....of the EM algorithm based on standard numerical tools to speed up the convergence. There are often e ective, but they do not guarantee monotone increase in the objective function. To overcome this problem, alternatives based on model reduction ( 22] 14] and e cient data augmentation ( 6] [7], 9] 20] 21] 23] 15] see also the chapter 5 of [18] have recently been considered. These extensions share the simplicity and stability with EM while speeding up the convergence. However, as far as we know, only two extensions ( 24] 16] were devoted to speeding up the convergence in ....

....describe our component wise algorithm and show, in Section 4, that it can also be interpreted as a proximal point algorithm. Using this interpretation, convergence of CEM 2 is proved in Section 5. Illustrative numerical experiments comparing the behaviors of EM, a version of the SAGE algorithm [6, 7] and CEM 2 are presented in Section 6. Concluding remarks end the paper. An appendix carefully describes the SAGE method in the mixture context in order to provide detailed comparison with the proposed CEM 2 . 2 EM type algorithms for mixtures We consider a J component mixture in R d g(yj ) ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Trans. Image Processing, 4:14171429, 1995.

No context found.

JA Fessler and AO Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, Oct. 1995.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized em algorithms," IEEE Trans. on Image Processing, vol. 4, no. 10, pp. 1417--29, 1995.

....efficiency and PET photon absorption survival probabilities) and g ij represents the geometric portion of the system matrix. I recommend estimating x by maximizing a penalized likelihood objective. The fastest monotonic algorithm I know for performing this maximization is the PML SAGE 3 algorithm [13], which I will document here along with the hideously slow ML EM algorithm for comparison. Typing i empl2 will show the arguments for this method: Usage: empl2 out init yi ci ri ri scale wtf mask method [saver flag obj(0) pix max scale init(0) slices ] Again, most of these are identical ....

....some are described below. 16 9.4.1 ML EM em,1 is the standard ML EM 1 algorithm (which is unregularized, so just use a for the penalty, i.e. 30 ml,1 would be the method argument for the ubiquitous 30 iterations of ML EM. 9.4. 2 SAGE sage,3,raster1 applies the PML SAGE 3 algorithm [13], with a large collection of penalty functions available. For now, I recommend the penalty choice quad,1,b2info since it gives more uniform resolution than conventional regularization methods [2] and works with the # tabulation described in Section 6. In the near future I hope to document other ....

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, October 1995.

.... decreases the objective function, i.e. Phi(x ) by showing that OE is a comparison function that satisfies ) Gamma OE ) Phi(x) Gamma Phi(x ) 13) This is precisely the property used to establish monotonicity of EM type algorithms that can be written in the form (11) [13 15]. The monotonicity property ensures that the sequence f Phi(x )g converges (since Phi is bounded below by i=1 i (0) However, Huber did not proceed to prove that the sequence of iterates fx g converges. Instead he established convergence of a somewhat different algorithm called the ....

....converges (since Phi is bounded below by i=1 i (0) However, Huber did not proceed to prove that the sequence of iterates fx g converges. Instead he established convergence of a somewhat different algorithm called the modified residuals method [4] Using arguments very similar to those in [15, 16], we have shown that if Phi has a unique minimizer x, then fx g converges globally to x. Strict convexity of Phi is sufficient but certainly not necessary to ensure Phi has a unique minimizer. The details will be provided elsewhere, hopefully after resolving whether the sequence ....

[Article contains additional citation context not shown here]

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, October 1995.

....proposed variance approximations we simulated 2000 realizations of PET emission scans using the emission phantom shown in Fig. 1. The simulation parameters are identical to those discussed in [7] From each sinogram realization we computed using 10 iterations of the PML SAGE 3 algorithm of [9] and the modified quadratic penalty of [7] We also computed the PL estimates for the standard quadratic penalty function (which has nonuniform spatial resolution [7] Both algorithms enforced the nonnegativity constraint. Presented at 1997 IEEE Nuc. Sci. Symp. and Med. Im. Conf. For each of ....

J. A. Fessler and A. O. Hero, "Penalized maximumlikelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417--29, October 1995.

....60 20 40 60 80 1 oo 120 Fig. 2. Digital phantom used for investigation of resolution properties of different regularizations, with four pixels of interest marked. order reduction of n to nG. For 2D reconstructions performed in the following sec tion, 30 iterations of the SAGE algorithm [20] on a 266 MHz Pentium II processor took 18.5 seconds for the conventional space invariant first order penalty given by the kernel in (5) and 20.1 seconds for the proposed penalt.y with precomputed J0 and M. The precalculation of 30 and M, took 23.1 seconds 11. Thus the method is very practical. ....

....standard deviation images, we simulated 400 noisy measurement realizations for the digital phantom in Fig. 2. The PET model included 10 random coincidences and averaged I million counts per realization. We reconstructed each of these 400 realizations using 30 iterations of the SAGE algorithm [20] with the same regu larization methods used above in the resolution properties investigation. For all of the statistical methods except the CNLLS penalty, we use the measurements, Yi, for calculation of . Because of the extensive computation time associated with calculation of the CNLLS penalty, ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms, " IEEE Tr. Ira. Proc., vol. 4, pp. 1417-29, Oct. 1995.

....efficiency and PET photon absorption survival probabilities) and g ij represents the geometric portion of the system matrix. I recommend estimating x by maximizing a penalized likelihood objective. The fastest monotonic algorithm I know for performing this maximization is the PML SAGE 3 algorithm [13], which I will document here along with the hideously slow ML EM algorithm for comparison. Typing i empl2 will show the arguments for this method: Usage: empl2 out init yi ci ri ri scale wtf mask method [saver flag obj(0) pix max scale init(0) slices ] Again, most of these are identical to ....

....the choices, since only some are described below. ML EM em,1 is the standard ML EM 1 algorithm (which is unregularized, so just use a forthepenalty, i.e. 30 ml,1 would be the method argument for the ubiquitous 30 iterations of ML EM. SAGE sage,3,raster1 applies the PML SAGE 3 algorithm [13], with a large collection of penalty functions available. For now, I recommend the penalty choice quad,1,b2info since it gives more uniform resolution than conventional regularization methods [2] and works with the # tabulation described in Section VI. In the near future I hope to document other ....

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, October 1995.

.... (ECME) 239] but SAGE generally has faster convergence [265] In Meng and Van Dyk s paper [265] a generalization of SAGE and ECME was introduced called alternating expectation conditional maximization (AECM) which is a SAGE algorithm with a design parameter similar to SAGE 3 introduced in [111]. Another recent generalization, called parameter expansion EM (PX EM) allows one to augment the parameter space, in addition to the data space, in order to obtain further convergence acceleration [240] Interestingly, for the case of the superimposed signals problem PX EM reduces to SAGE. In ....

....to augment the parameter space, in addition to the data space, in order to obtain further convergence acceleration [240] Interestingly, for the case of the superimposed signals problem PX EM reduces to SAGE. In addition to the examples shown in Submitted to IEEE SP Magazine, June 1998 8 [110] [111] and [265] the SAGE algorithm and its variants have been applied to: angle of arrival estimation [115] multi user detection [289, 81] estimation of constrained covariance matrices [364] and speckle interferometry [363] For cases where the likelihood function is non convex neither the EM ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space alternating generalized EM algorithm," IEEE Trans. on Image Processing, vol. IP-4, no. 10, pp. 1417--1429, Oct. 1995.

....iteration [54] If the penalty term is convex, another approach by De Pierro [24] uses convexity to find separable surrogates for the penalty part which decouples the M step and yields a closed form update. Alternatively, by using a separate hidden data space for each parameter, the SAGE algorithm [44, 45] intrinsically uncouples the parameter updates and uses less informative complete data spaces to improve convergence speed. These algorithms are provably convergent algorithms for emission tomography. The least informative data space is the measurement space itself. Thus, some researchers worked ....

....for obtaining better curvatures. 4.3.4 Convergence and Convergence Rate In the absence of background events, i.e. when r i = 0, the penalized likelihood objective # is convex and our proposed PSCD algorithm is globally convergent. This is a fairly straightforward consequence of the proof in [45] for convergence of SAGE, so we omit the details. However when r i #= 0, little can be said about global convergence due to the possibility that there are multiple minima or a continuous region of minima. Our practical experience suggests that local minima are either unlikely to be present, or ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc.,vol.4,no. 10, pp. 1417--29, October 1995.

.... thus one needs to include a line search [60] Although conjugate gradient methods have rapid convergence for quadratic optimization, usually one needs some form of preconditioner and enforcing non negativity of the solution is possible but di#cult [68] Space alternating generalized EM (SAGE) [40 42] is a generalized EM type algorithm which updates parameters sequentially by alternating between small hidden data spaces [41] As SAGE uses separate hidden data spaces for each parameter, not only the maximization is simplified but convergence rate is also improved compared to EM. SAGE was shown ....

....the conditional expectation of the complete data space and simultaneously maximizes the expectation with respect to unknown parameters. Since its introduction, 61, 81] EM method has been used widely to compute ML estimates in emission tomography. Space alternating generalized EM (SAGE) algorithm [40 42] is a generalized EM type algorithm which updates parameters sequentially by alternating between small hidden data spaces [41] As SAGE uses separate hidden data spaces for each parameter, not only the maximization is simplified but convergence rate is also improved compared to EM. In this section ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417-- 29, October 1995.

....emission images from Poisson sinogram measurements. The algorithm . monotonically increases the objective function, is globally convergent, naturally accommodates the nonnegativity constraint, requires less CPU time per iteration than the spacealternating generalize EM (SAGE) algorithm [1]. For reconstruction problems (such as 2D PET and SPECT) where the system matrix G can be precomputed and stored, we recommend this new algorithm over our previously published algorithms for penalized likelihood reconstruction. For reconstruction problems where the system matrix is represented in ....

....Function (l) and q i (l; l n i ) Log likelihood h i (l) Surrogate q i (l; l n i ) l n i l # Figure 1: Illustration of 1D parabolic surrogate function. Note that q i (l; l n i ) # h i (l) for l # 0. III. THE NEW ALGORITHM We had previously recommended the SAGE algorithm [1] as a fast globally convergent algorithm for this problem. The construction of the SAGE algorithm requires certain minimizations that are somewhat unusual in the tomographic literature, and must be implemented carefully to achieve reasonable CPU time per iteration. The new algorithm we propose is ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero, "Penalized maximumlikelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc.,vol.4, no. 10, pp. 1417--29, October 1995.

....desirable to quickly recover degraded images. Simultaneously updating all parameters in the EM algorithms [1, 2] causes a very slow convergence rate. Therefore a number of algorithms have been proposed to increase the convergence rate such as the space alternating generalized EM (SAGE) algorithm [3, 4]. The SAGE method converges quickly but it is inconvenient to implement. Due to slow convergence in simultaneous updates, algorithms based on sequential updates, such as a coordinate ascent algorithm with Newton Raphson updates (CA NR) 5] have become attractive. However, the CA NR algorithm is ....

J. A. Fessler and A. O. Hero, "Penalized MaximumLikelihood Image Reconstruction Using SpaceAlternating Generalized EM Algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417-- 1429, October 1995.

....and a deconvolve shrink estimator. We maximized the nonquadratic penalized likelihood objective using the PMLSAGE algorithm, a variant of the iterative space alternating generalized expectation maximization (SAGE) algorithm of [20] adapted for penalized maximum likelihood image reconstruction [46]. We initialized PML SAGE with an unweighted penalized least squares estimate: A T A I) 1 A 0 (Y r) which is linear so can be computed noniteratively. Here = P j A jk = P j A jk =Y j ) for k = 65 (cf [47] 48] By so initializing, only 30 iterations were needed to ....

J. A. Fessler and A. O. Hero, \Penalized maximum likelihood image reconstruction using space alternating generalized EM algorithms, " IEEE Trans. on Image Processing, vol. 3, , to appear Oct. 1995.

No context found.

J. Fessler and A. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 1417--1429, 1995.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithm," IEEE Trans. Imag. Process. 4, 10 (1995) 1417--1429.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417--1429, 1995.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417--1429, 1995.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417--1429, 1995.

No context found.

J. A. Fessler and A. O. Hero, \Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417-1429, 1995.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC