J. Fessler and A. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 1417--1429, 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... with convergence speed in iteration counts which is very similar to direct ICD [10] One may also solve the EM formulation pixel sequentially, preserving the provable convergence of EM while substantially improving its speed, as in the space alternating EM algorithm (SAGE) of Fessler and Hero [15]. Column action methods can easily be made to converge reliably to the unique global minimum of the ML or MAP functional. Here we present two improvements to current forms of ICD: 1) global convergence of the approximate greedy descent algorithm follows from the introduction of a new local ....

....can be efficiently implemented via table look ups. B. Global Convergence of ICD FS In order to prove the global convergence of this new ICD FS algorithm, we simply verify that it meets the assumptions and necessary conditions of the global convergence proof presented by Fessler and Hero in [15] for convergence of SAGE under positivity constraints. Since this proof requires continuity of the log likelihood on , we must assume that the background noise is greater than zero, i.e. in emission case (1) We discuss alternative methods for the case later in this section. a) b) c) d) ....

[Article contains additional citation context not shown here]

J. Fessler and A. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 1417--1429, Oct. 1995.

....of the EM algorithm based on standard numerical tools to speed up the convergence. There are often effective, but they do not guarantee monotone increase in the objective function. To overcome this problem, alternatives based on model reduction ( 22] 14] and efficient data augmentation ( 6] [7], 9] 20] 21] 23] 15] see also the chapter 5 of [18] have recently been considered. These extensions share the simplicity and stability with EM while speeding up the convergence. However, as far as we know, only two extensions ( 24] 16] were devoted to speeding up the convergence in ....

....3, we describe our component wise algorithm and show, in Section 4, that it can also be interpreted as a proximal point algorithm. Using this interpretation, convergence of CEM proved in Section 5. Illustrative numerical experiments comparing the behaviors of EM, a version of the SAGE algorithm [6, 7] and CEM are presented in Section 6. A discussion section ends the paper. An appendix carefully describes the SAGE method in the mixture context in order to provide detailed comparison with the proposed . 2 EM type algorithms for mixtures We consider a J component mixture in R d g(yj ) ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Trans. Image Processing, 4:1417--1429, 1995.

....Another interesting recent method is the one in [27] The methods of [13] 27] 25] 30] constitute the state of the art. Finally, we mention that EM and EM type algorithms have been previously used in image restoration and reconstruction, although not in a wavelet based formulation (e.g. [9], 10] 15] V. The Best of Both Worlds The approach proposed in this paper is able to use the best of the wavelet and Fourier worlds in image deconvolution problems. The speed and convenience of FFT based linear ltering, which is well matched to the observation model, and the adequacy of ....

J. Fessler, A. Hero. \Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. on Image Processing, vol. 4, pp. 1417-29, 1995.

....and Operations Research Department, George Mason University, Fairfax, VA 22030 4444 USA. Publisher Item Identifier S 0278 0062(00)02978 5. facts in later iterates of the expectation maximization (EM) algorithm [54] Regularization of the objective function is commonly referred to as penalized ML [10] or maximum a posteriori (MAP) reconstruction [35] and has generated significant interest because, in addition to improving the noise properties of the converged solution, inclusion of a regularization term in the objective function also leads to faster convergence [33] PET reconstructions of ....

.... of nonnegativity via a quadratic penalty term in the objective function [40] and in a separate paper explored active set methods [41] Coordinate ascent and grouped coordinate ascent methods have recently been reported to improve convergence significantly over various MAP EM algorithms [2] [10], 11] 20] While the results presented in these latter papers are quite strong, GaussSeidel iterations do not accommodate convenient system matrix factorizations such as those in [40] This paper introduces ML reconstruction methods that solve a sequence of subproblems that successively ....

J. A. Fessler, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 14101429, Oct., 1995.

.... uniquely existent and is termed as the accelerated cross reference maximum likelihood estimate (ACRMLE) New algorithms adapted from the expectation conditional maximization (ECM) algorithm in [14] as well as the space alternating generalized expectation maximization (SAGE) algorithms in [15] and [16] are developed in this article. These algorithms are shown to be convergent and have fast convergence speeds. Simulation studies reveal that an even faster convergence speed is achieved if one alternates the order of iterations. The penalty parameter can be selected by users or data driven ....

....by Theorem 2, Phi( new ; new d ) Phi( old ; old d ) According to Theorem 4, the modified ECM sequence ( t) t) d ) obtained from Algorithm 1 converges to the unique maximizer, ACRMLE1 ; d;ACRMLE1 ) C. The Modified SAGE Algorithm Fessler and Hero in [15] and [16] proposed another iterative methods to accelerate the convergence speed of the EM algorithm. They investigated a different approach in specifying the hidden data space such that the resulting space alternating generalized expectation maximization algorithm (SAGE) has a faster convergence speed ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized expectation-maximization algorithms," IEEE Trans. on Image Processing, vol. 4, no. 10, pp. 1417--1429, 1995.

....of the EM algorithm based on standard numerical tools to speed up the convergence. There are often e ective, but they do not guarantee monotone increase in the objective function. To overcome this problem, alternatives based on model reduction ( 22] 14] and e cient data augmentation ( 6] [7], 9] 20] 21] 23] 15] see also the chapter 5 of [18] have recently been considered. These extensions share the simplicity and stability with EM while speeding up the convergence. However, as far as we know, only two extensions ( 24] 16] were devoted to speeding up the convergence in ....

....describe our component wise algorithm and show, in Section 4, that it can also be interpreted as a proximal point algorithm. Using this interpretation, convergence of CEM 2 is proved in Section 5. Illustrative numerical experiments comparing the behaviors of EM, a version of the SAGE algorithm [6, 7] and CEM 2 are presented in Section 6. Concluding remarks end the paper. An appendix carefully describes the SAGE method in the mixture context in order to provide detailed comparison with the proposed CEM 2 . 2 EM type algorithms for mixtures We consider a J component mixture in R d g(yj ) ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Trans. Image Processing, 4:14171429, 1995.

....increase in the objective function. To overcome this problem, alternatives based on model reduction and missing data space reduction have recently been considered. Model reduction is illustrated in Meng and Rubin (1993) and Liu and Rubin (1994) while missing data space reduction is the object of Fessler and Hero (1994, 1995), Hero and Fessler (1995) Meng and van Dyk (1997, 1998) Neal and Hinton (1998) Liu, Rubin and Wu (1998) and also chapter 5 of McLachlan (1997) These extensions generally share the simplicity and stability of EM while speeding up the convergence, although the ECM algorithm of Meng and Rubin ....

....this problem, alternatives based on model reduction and missing data space reduction have recently been considered. Model reduction is illustrated in Meng and Rubin (1993) and Liu and Rubin (1994) while missing data space reduction is the object of Fessler and Hero (1994, 1995) Hero and Fessler (1995), Meng and van Dyk (1997, 1998) Neal and Hinton (1998) Liu, Rubin and Wu (1998) and also chapter 5 of McLachlan (1997) These extensions generally share the simplicity and stability of EM while speeding up the convergence, although the ECM algorithm of Meng and Rubin (1993) does not necessarily ....

[Article contains additional citation context not shown here]

Fessler, J. A., and Hero, A. O. (1995), "Penalized maximum-likelihood image reconstruction using spaceAlternating generalized EM algorithms", IEEE Trans. Image Processing, 4, 1417-1429.

.... with convergence speed in iteration counts which is very similar to direct ICD[10] One may also solve the EM formulation pixel sequentially, preserving the provable convergence of EM while substantially improving its speed, as in the space alternating EM algorithm (SAGE) of Fessler and Hero [15]. Column action methods can easily be made to converge reliably to the unique global minimum of the ML or MAP functional. Here we present two improvements to current forms of ICD: 1) global convergence of the approximate greedy descent algorithm follows from the introduction of a new local ....

....can be e#ciently implemented via table look ups. 3. 2 Global Convergence of ICD FS In order to prove the global convergence of this new ICD FS algorithm, we simply verify that it meets the assumptions and necessary conditions of the global convergence proof presented by Fessler and Hero in [15] for convergence of SAGE under positivity constraints. Since this proof requires continuity of the log likelihood on IR N , we must assume that the background noise is greater than zero, i.e. r i 0 in emission case (1) We discuss alternative methods for the case r i = 0 later in this ....

[Article contains additional citation context not shown here]

J. Fessler and A. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. on Image Processing, vol. 4, no. 10, pp. 1417--1429, October 1995.

....than the MRP OS EM, if the latter is left iterate a little longer. The MRP OS ISRA technique shows increased streak artifacts and blurs the body and ROI contours. The MRP OS SAGE is not providing any visible improvement over the MRP OS EM. More recent modifications of the SAGE algorithm (SAGE 3) 19] might provide better acceleration factors and image quality. The error images in Fig. 2 verify these observations (quantitative difference of the MRP OS WLS, MRP OS ISRA and MRPOS SAGE images from the MRP OS EM one) Figure 3: The same slice shown in Fig. 2,after 24 iterations without OS ....

J. A. Fessler, A. Hero, "Penalized maximum--likelihood image reconstruction using space-alternating generalized EM algorithms", IEEE Trans Imag Proc, 4:1417-1429, 1995.

.... associated with traditional EM based algorithms [12] For problems in quantum limited imaging and the estimation of superimposed signals in Gaussian noise, these differences have been shown to result in algorithms that converge much faster than algorithms derived by conventional EM procedures [12,20]. As with the EM procedure, algorithms derived by applying the SAGE procedure have the desirable property of producing a sequence of parameter estimates that are intrinsically monotonic in likelihood [20] In contrast with the EM procedure wherein the parameter dependencies are updated only after ....

.... algorithms that converge much faster than algorithms derived by conventional EM procedures [12,20] As with the EM procedure, algorithms derived by applying the SAGE procedure have the desirable property of producing a sequence of parameter estimates that are intrinsically monotonic in likelihood [20]. In contrast with the EM procedure wherein the parameter dependencies are updated only after all parameters have been updated, the structure for an algorithm derived via the SAGE procedure can be of the following form: ffl initialize parameters and their dependencies ffl Iterate: for i = 1; 2; ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms", IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417--1429, 1995.

.... with convergence speed in iteration counts which is very similar to direct ICD[10] One may also solve the EM formulation pixel sequentially, preserving the provable convergence of EM while substantially improving its speed, as in the space alternating EM algorithm (SAGE) of Fessler and Hero [15]. Column action methods can easily be made to converge reliably to the unique global minimum of the ML or MAP functional. Here we present two improvements to current forms of ICD: 1) global convergence of the approximate greedy descent algorithm follows from the introduction of a new local ....

....can be efficiently implemented via table look ups. 3. 1 Global Convergence of ICD FS In order to prove the global convergence of this new ICD FS algorithm, we simply verify that it meets the two assumptions and six conditions of the global convergence proof presented by Fessler and Hero in [15] for convergence of SAGE. Since this proof requires continuity of the log likelihood on IR N , we must assume that the background noise is greater than zero, i.e. r i 0 in emission case (1) We discuss alternative methods for the case r i = 0 later in this section. Most of these conditions ....

[Article contains additional citation context not shown here]

J. Fessler and A. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. on Image Processing, vol. 4, no. 10, pp. 1417--1429, October 1995.

No context found.

JA Fessler and AO Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, Oct. 1995.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized em algorithms," IEEE Trans. on Image Processing, vol. 4, no. 10, pp. 1417--29, 1995.

....efficiency and PET photon absorption survival probabilities) and g ij represents the geometric portion of the system matrix. I recommend estimating x by maximizing a penalized likelihood objective. The fastest monotonic algorithm I know for performing this maximization is the PML SAGE 3 algorithm [13], which I will document here along with the hideously slow ML EM algorithm for comparison. Typing i empl2 will show the arguments for this method: Usage: empl2 out init yi ci ri ri scale wtf mask method [saver flag obj(0) pix max scale init(0) slices ] Again, most of these are identical ....

....some are described below. 16 9.4.1 ML EM em,1 is the standard ML EM 1 algorithm (which is unregularized, so just use a for the penalty, i.e. 30 ml,1 would be the method argument for the ubiquitous 30 iterations of ML EM. 9.4. 2 SAGE sage,3,raster1 applies the PML SAGE 3 algorithm [13], with a large collection of penalty functions available. For now, I recommend the penalty choice quad,1,b2info since it gives more uniform resolution than conventional regularization methods [2] and works with the # tabulation described in Section 6. In the near future I hope to document other ....

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, October 1995.

.... decreases the objective function, i.e. Phi(x ) by showing that OE is a comparison function that satisfies ) Gamma OE ) Phi(x) Gamma Phi(x ) 13) This is precisely the property used to establish monotonicity of EM type algorithms that can be written in the form (11) [13 15]. The monotonicity property ensures that the sequence f Phi(x )g converges (since Phi is bounded below by i=1 i (0) However, Huber did not proceed to prove that the sequence of iterates fx g converges. Instead he established convergence of a somewhat different algorithm called the ....

....converges (since Phi is bounded below by i=1 i (0) However, Huber did not proceed to prove that the sequence of iterates fx g converges. Instead he established convergence of a somewhat different algorithm called the modified residuals method [4] Using arguments very similar to those in [15, 16], we have shown that if Phi has a unique minimizer x, then fx g converges globally to x. Strict convexity of Phi is sufficient but certainly not necessary to ensure Phi has a unique minimizer. The details will be provided elsewhere, hopefully after resolving whether the sequence ....

[Article contains additional citation context not shown here]

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, October 1995.

....proposed variance approximations we simulated 2000 realizations of PET emission scans using the emission phantom shown in Fig. 1. The simulation parameters are identical to those discussed in [7] From each sinogram realization we computed using 10 iterations of the PML SAGE 3 algorithm of [9] and the modified quadratic penalty of [7] We also computed the PL estimates for the standard quadratic penalty function (which has nonuniform spatial resolution [7] Both algorithms enforced the nonnegativity constraint. Presented at 1997 IEEE Nuc. Sci. Symp. and Med. Im. Conf. For each of ....

J. A. Fessler and A. O. Hero, "Penalized maximumlikelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417--29, October 1995.

....60 20 40 60 80 1 oo 120 Fig. 2. Digital phantom used for investigation of resolution properties of different regularizations, with four pixels of interest marked. order reduction of n to nG. For 2D reconstructions performed in the following sec tion, 30 iterations of the SAGE algorithm [20] on a 266 MHz Pentium II processor took 18.5 seconds for the conventional space invariant first order penalty given by the kernel in (5) and 20.1 seconds for the proposed penalt.y with precomputed J0 and M. The precalculation of 30 and M, took 23.1 seconds 11. Thus the method is very practical. ....

....standard deviation images, we simulated 400 noisy measurement realizations for the digital phantom in Fig. 2. The PET model included 10 random coincidences and averaged I million counts per realization. We reconstructed each of these 400 realizations using 30 iterations of the SAGE algorithm [20] with the same regu larization methods used above in the resolution properties investigation. For all of the statistical methods except the CNLLS penalty, we use the measurements, Yi, for calculation of . Because of the extensive computation time associated with calculation of the CNLLS penalty, ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms, " IEEE Tr. Ira. Proc., vol. 4, pp. 1417-29, Oct. 1995.

....efficiency and PET photon absorption survival probabilities) and g ij represents the geometric portion of the system matrix. I recommend estimating x by maximizing a penalized likelihood objective. The fastest monotonic algorithm I know for performing this maximization is the PML SAGE 3 algorithm [13], which I will document here along with the hideously slow ML EM algorithm for comparison. Typing i empl2 will show the arguments for this method: Usage: empl2 out init yi ci ri ri scale wtf mask method [saver flag obj(0) pix max scale init(0) slices ] Again, most of these are identical to ....

....the choices, since only some are described below. ML EM em,1 is the standard ML EM 1 algorithm (which is unregularized, so just use a forthepenalty, i.e. 30 ml,1 would be the method argument for the ubiquitous 30 iterations of ML EM. SAGE sage,3,raster1 applies the PML SAGE 3 algorithm [13], with a large collection of penalty functions available. For now, I recommend the penalty choice quad,1,b2info since it gives more uniform resolution than conventional regularization methods [2] and works with the # tabulation described in Section VI. In the near future I hope to document other ....

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, October 1995.

.... (ECME) 239] but SAGE generally has faster convergence [265] In Meng and Van Dyk s paper [265] a generalization of SAGE and ECME was introduced called alternating expectation conditional maximization (AECM) which is a SAGE algorithm with a design parameter similar to SAGE 3 introduced in [111]. Another recent generalization, called parameter expansion EM (PX EM) allows one to augment the parameter space, in addition to the data space, in order to obtain further convergence acceleration [240] Interestingly, for the case of the superimposed signals problem PX EM reduces to SAGE. In ....

....to augment the parameter space, in addition to the data space, in order to obtain further convergence acceleration [240] Interestingly, for the case of the superimposed signals problem PX EM reduces to SAGE. In addition to the examples shown in Submitted to IEEE SP Magazine, June 1998 8 [110] [111] and [265] the SAGE algorithm and its variants have been applied to: angle of arrival estimation [115] multi user detection [289, 81] estimation of constrained covariance matrices [364] and speckle interferometry [363] For cases where the likelihood function is non convex neither the EM ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space alternating generalized EM algorithm," IEEE Trans. on Image Processing, vol. IP-4, no. 10, pp. 1417--1429, Oct. 1995.

....iteration [54] If the penalty term is convex, another approach by De Pierro [24] uses convexity to find separable surrogates for the penalty part which decouples the M step and yields a closed form update. Alternatively, by using a separate hidden data space for each parameter, the SAGE algorithm [44, 45] intrinsically uncouples the parameter updates and uses less informative complete data spaces to improve convergence speed. These algorithms are provably convergent algorithms for emission tomography. The least informative data space is the measurement space itself. Thus, some researchers worked ....

....for obtaining better curvatures. 4.3.4 Convergence and Convergence Rate In the absence of background events, i.e. when r i = 0, the penalized likelihood objective # is convex and our proposed PSCD algorithm is globally convergent. This is a fairly straightforward consequence of the proof in [45] for convergence of SAGE, so we omit the details. However when r i #= 0, little can be said about global convergence due to the possibility that there are multiple minima or a continuous region of minima. Our practical experience suggests that local minima are either unlikely to be present, or ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc.,vol.4,no. 10, pp. 1417--29, October 1995.

.... thus one needs to include a line search [60] Although conjugate gradient methods have rapid convergence for quadratic optimization, usually one needs some form of preconditioner and enforcing non negativity of the solution is possible but di#cult [68] Space alternating generalized EM (SAGE) [40 42] is a generalized EM type algorithm which updates parameters sequentially by alternating between small hidden data spaces [41] As SAGE uses separate hidden data spaces for each parameter, not only the maximization is simplified but convergence rate is also improved compared to EM. SAGE was shown ....

....the conditional expectation of the complete data space and simultaneously maximizes the expectation with respect to unknown parameters. Since its introduction, 61, 81] EM method has been used widely to compute ML estimates in emission tomography. Space alternating generalized EM (SAGE) algorithm [40 42] is a generalized EM type algorithm which updates parameters sequentially by alternating between small hidden data spaces [41] As SAGE uses separate hidden data spaces for each parameter, not only the maximization is simplified but convergence rate is also improved compared to EM. In this section ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417-- 29, October 1995.

....emission images from Poisson sinogram measurements. The algorithm . monotonically increases the objective function, is globally convergent, naturally accommodates the nonnegativity constraint, requires less CPU time per iteration than the spacealternating generalize EM (SAGE) algorithm [1]. For reconstruction problems (such as 2D PET and SPECT) where the system matrix G can be precomputed and stored, we recommend this new algorithm over our previously published algorithms for penalized likelihood reconstruction. For reconstruction problems where the system matrix is represented in ....

....Function (l) and q i (l; l n i ) Log likelihood h i (l) Surrogate q i (l; l n i ) l n i l # Figure 1: Illustration of 1D parabolic surrogate function. Note that q i (l; l n i ) # h i (l) for l # 0. III. THE NEW ALGORITHM We had previously recommended the SAGE algorithm [1] as a fast globally convergent algorithm for this problem. The construction of the SAGE algorithm requires certain minimizations that are somewhat unusual in the tomographic literature, and must be implemented carefully to achieve reasonable CPU time per iteration. The new algorithm we propose is ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero, "Penalized maximumlikelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc.,vol.4, no. 10, pp. 1417--29, October 1995.

....desirable to quickly recover degraded images. Simultaneously updating all parameters in the EM algorithms [1, 2] causes a very slow convergence rate. Therefore a number of algorithms have been proposed to increase the convergence rate such as the space alternating generalized EM (SAGE) algorithm [3, 4]. The SAGE method converges quickly but it is inconvenient to implement. Due to slow convergence in simultaneous updates, algorithms based on sequential updates, such as a coordinate ascent algorithm with Newton Raphson updates (CA NR) 5] have become attractive. However, the CA NR algorithm is ....

J. A. Fessler and A. O. Hero, "Penalized MaximumLikelihood Image Reconstruction Using SpaceAlternating Generalized EM Algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417-- 1429, October 1995.

....and a deconvolve shrink estimator. We maximized the nonquadratic penalized likelihood objective using the PMLSAGE algorithm, a variant of the iterative space alternating generalized expectation maximization (SAGE) algorithm of [20] adapted for penalized maximum likelihood image reconstruction [46]. We initialized PML SAGE with an unweighted penalized least squares estimate: A T A I) 1 A 0 (Y r) which is linear so can be computed noniteratively. Here = P j A jk = P j A jk =Y j ) for k = 65 (cf [47] 48] By so initializing, only 30 iterations were needed to ....

J. A. Fessler and A. O. Hero, \Penalized maximum likelihood image reconstruction using space alternating generalized EM algorithms, " IEEE Trans. on Image Processing, vol. 3, , to appear Oct. 1995.

....(31) The results of (31) are not shown in Fig. 4 since they turn out to be indistinguishable from the curves shown, which supports the accuracy of the approximations leading to (31) We maximized the objective function (15) to compute in (5) using 20 iterations of the PML SAGE 3 algorithm [18]. Fig. 4 displays horizontal and vertical profiles through the local impulse responses for the estimators correspond 11 ing to the two penalty functions. The circles in Fig. 4 are for the unbiased estimator (6) for M = 2000 realizations. The standard penalty has highly nonuniform spatial ....

....tradeoff To address this question, we generated 100 realizations of Poisson distributed simulated PET measurements for the object shown in Fig. 3, and for the system properties described in Section VI. For each realization y (1) y (100) we used 20 iterations of PML SAGE 3 [18] to compute penalized likelihood estimates f (y (m) g 100 m=1 for several values of fi for both the standard and the modified quadratic penalties. For each value of fi, we computed 12 40 45 50 28 30 32 34 36 80 85 90 28 30 32 34 36 40 45 50 28 30 32 34 36 80 85 90 28 30 ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms, " IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417--29, Oct. 1995.

....i.e. j Phi( 0; j 0 j Phi( 0; j = 0 ; and if Phi is strictly concave in a neighborhood of , then is a fixed point of (7) and using continuity one can prove local convergence of the iteration (7) to . For reasons of convergence rate [25,26,27], we usually use = 0:6. This under relaxation improves the odds that (7) will yield an increase in Phi. With = 0:6 we have never observed a decrease in Phi over a full iteration, although we have observed small decreases with = 1. Fortuitously, using 1 not only improves the convergence ....

J. A. Fessler. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Trans. Im. Proc., 1995. Accepted.

....motivation for obtaining better curvatures. D. Convergence and Convergence Rate In the absence of background events, i.e. when , the penalized likelihood objective is convex and our proposed PSCD algorithm is globally convergent. This is a fairly straightforward consequence of the proof in [25] for the convergence of SAGE, so we omit the details. However when , little can be said about global convergence due to the possibility that there are multiple minima or a continuous region of minima. Our practical experience suggests that local minima are either unlikely to be present, or are ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417--1429, Oct. 1995.

....in the object except across the organ boundary. This leads to an iterative penalized maximum likelihood (PML) reconstruction algorithm, equivalently a MAP algorithm with Gibbs prior, implemented using the space alternating generalized expectation maximization (SAGE EM) version of the EM algorithm [19]. In Section IV C the PML algorithm with perfect side information is generalized to the case of noisy boundary estimates. Here the MRI derived boundary estimator resolution may be lower than ECT spatial resolution in some regions of the image. To deal with imperfect side information a minimax ....

....the ML algorithm suffers from slow convergence and is not well suited to incorporation of image smoothness or anatomical side information. Penalized ML (PML) image reconstruction can enforce image smoothness by introducing roughness penalty functions into the log likelihood objective function [19], 34] The PML reconstruction is the maximizer over of (14) Here, is the loglikelihood function of given the ECT measurements is a vector of weights, is a quadratic penalty (15) and is a smoothness parameter a large value of strongly emphasizes the penalty, and hence encourages smoothness in ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. Hero, "Penalized maximum likelihood image reconstruction using space alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 1417--1429, Oct. 1995.

....at least O(2M 2 p 2 log p] floating point operations to evaluate r j for all j. Clearly, much of the computational advantage of the proposed method is due to the order reduction of M 2 to M . For 2D reconstructions performed in the following section, 30 iterations of the SAGE algorithm [15] on a 266 MHz Pentium II processor took 18.5 seconds for the conventional space invariant first order penalty given by the kernel in (6) and 20.1 seconds for the proposed penalty with precomputed # j0 and M . The precalculation of # j0 and M , which only need be done once for a given system ....

....standard deviation images, we simulated 400 noisy measurement realizations for the digital phantom in Fig. 2. The PET model included 10 random coincidences and averaged 1 million counts per realization. We reconstructed each of these 400 realizations using 30 iterations of the SAGE algorithm [15] with the same regularization methods used above in the resolution properties investigation. For all of the statistical methods except the CNLLS penalty, we use the measurements, y i , for calculation of R. Because of the extensive computation time associated with calculation of the CNLLS penalty, ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalizedEM algorithms," IEEE Tr. Im. Proc., vol. 4, pp. 1417--29, Oct. 1995.

....ML estimate of a = a; d; b 0 ; b N Gamma1 ] T is defined as = arg max 2 Theta flog p(Y ; Gamma P ( g : 17) Here P ( is a user specified penalty function. Penalties have frequently been introduced to regularize the estimator [40] to promote faster convergence [14], or to take advantage of prior information [39] It will be convenient to express the penalty function as the function P (z) Gamma log Pi(z) of the complex variable z = e j 0d defined in (13) In this way it can be seen that incorporation of the aforementioned polynomial rooting estimation ....

J.A. Fessler, A.O. Hero, "Penalized maximum-likelihood image reconstruction using spacealternating generalized EM algorithms", IEEE Trans. IP, 4(10):1417-1429, Oct. 1995.

....detector efficiency and PET photon absorption survival probabilities) and g ij represents the geometric portion of the system matrix. I recommend estimating by maximizing a penalized likelihood objective. The fastest algorithm I know for performing this maximization is the PML SAGE 3 algorithm [9], which I will document here along with the hideously slow ML EM algorithm for comparison. Typing i empl2 will show the arguments for this method: Usage: empl2 out init prompt calib backg backscale wtf mask method [saver flag obj(0) pix max scale init(0) slices ] Again, most of these are ....

....There are two algorithms implemented: em,1 is the standard ML EM 1 algorithm (which is unregularized, so just use a for the penalty, i.e. 30 ml,1 would be the method argument for the ubiquitous 30 iterations of ML EM. The other option is sage,3,raster1 which is the PML SAGE 3 algorithm [9], for which the following penalties are currently implemented: quad,1, quad,1,b2info I recommend the 2nd choice since it gives more uniform resolution, and works with the fi tabulation. There may also be nonquadratic penalties ask me if interested. This implementation assumes that r i 0. If ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417--29, October 1995.

.... slow convergence of the EM algorithm remains, and hundreds to thousands of EM iterations may be required for the post smoothed images to converge [26] This problem has spawned a variety of methods for accelerating the EM algorithm, which vary in the extent to which convergence is guaranteed, see [27,28]. Another disadvantage of the usual form of space invariant post smoothing is that the nonstationary measurement statistics are not incorporated. E. Classical Regularization Methods Another way to overcome the problems of slow convergence and to reduce the image noise is to replace the ....

....with less noise. When first investigated for PET, the penalty function posed a computational challenge since the M step of the EM algorithm has no closed form [29 31] However, now there are a variety of fast algorithms (compared to EM) available for maximizing such objective functions, e.g. [21, 27, 28, 32, 33]. These algorithms converge rapidly in part because the penalty function greatly improves the conditioning of the reconstruction problem. In the context of least squares problems, such regularization methods date at least to the early 70 s [34] so now may well be considered classical. The most ....

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, October 1995.

....in the object except across the organ boundary. This leads to an iterative penalized maximum likelihood (PML) reconstruction algorithm, equivalently a MAP algorithm with Gibbs prior, implemented using the space alternating generalized expectation maximization (SAGE EM) version of the EM algorithm [19]. In Section V C the PML algorithm with perfect side information is generalized to the case of noisy boundary estimates. Here the MRI derived boundary estimator resolution may be lower than ECT spatial resolution in some regions of the image. To deal with imperfect side information a minimax ....

....the ML algorithm suffers from slow convergence and is not well suited to incorporation of image smoothness or anatomical side information. Penalized ML (PML) image reconstruction can enforce image smoothness by introducing roughness penalty functions into the loglikelihood objective function [19], 35] The PML reconstruction is the maximizer over of Phi PML ( w) ln f(YE ; Gamma fiR( w) 18) Here, ln f(YE ; is the loglikelihood function of given the ECT measurements YE , w is a vector of weights, R( w) is a quadratic penalty R( w) P X j=1 X k2N j w jk Delta ( j ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. Hero, "Penalized maximum likelihood image reconstruction using space alternating generalized EM algorithms," IEEE Transactions on Image Processing, vol. 4, no. 10, pp. 1417--1429, October 1995.

....estimates of kinetic parameters from dynamic PET scans [27] VI. Example: Emission Tomography In this section we examine the accuracy of both the mean and the variance approximations for the problem of emission tomography. Our description of the problem is brief, for more details see [21] [28]. In emission tomography the parameter j denotes the radionuclide concentration in the jth pixel. The emission measurements have independent Poisson distributions, and we assume the mean of Yn is: Yn ( Tpn ( pn ( X j a nj j r n ; 29) where the a nj are proportional to the ....

....12.8 million counts. The r n factors were set to a uniform value corresponding to 10 random coincidences. For each study, 100 realizations of pseudo random Poisson transmission measurements were generated according to (29) and then reconstructed using a space alternating generalized EM algorithm [28], which enforces the nonnegativity constraint 0. FBP images served as the initial estimate for the iterative algorithm. For the penalty function OE we studied two cases: the simple quadratic case OE(x) x 2 =2, as well as a nonquadratic penalty: the third entry in Table III of [29] OE(x) ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms, " IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417--29, Oct. 1995.

....IMAGING, SUBMITTED February 10, 1999 8 D. Convergence and Convergence Rate In the absence of background events, i.e. when r i = 0, the penalized likelihood objective Phi is convex and our proposed PSCD algorithm is globally convergent. This is a fairly straightforward consequence of the proof in [25] for convergence of SAGE, so we omit the details. However when r i 6= 0, little can be said about global convergence due to the possibility that there are multiple minima or a continuous region of minima. Our practical experience suggests that local minima are either unlikely to be present, or are ....

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417--29, October 1995.

.... ) Phi(x n ) by showing that OE WLS is a comparison function that satisfies OE WLS (x; x n ) Gamma OE WLS (x n ; x n ) Phi(x) Gamma Phi(x n ) 13) This is precisely the property used to establish monotonicity of EM type algorithms that can be written in the form (11) [13 15]. The monotonicity property ensures that the sequence f Phi(x n )g converges (since Phi is bounded below by P m i=1 i (0) However, Huber did not proceed to prove that the sequence of iterates fx n g converges. Instead he established convergence of a somewhat different algorithm called ....

....(since Phi is bounded below by P m i=1 i (0) However, Huber did not proceed to prove that the sequence of iterates fx n g converges. Instead he established convergence of a somewhat different algorithm called the modified residuals method [4] Using arguments very similar to those in [15, 16], we have shown that if Phi has a unique minimizer x, then fx n g converges globally to x. Strict convexity of Phi is sufficient but certainly not necessary to ensure Phi has a unique minimizer. The details will be provided elsewhere, hopefully after resolving whether the sequence ....

[Article contains additional citation context not shown here]

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, October 1995.

....proposed variance approximations we simulated 2000 realizations of PET emission scans using the emission phantom shown in Fig. 1. The simulation parameters are identical to those discussed in [7] From each sinogram realization we computed using 10 iterations of the PML SAGE 3 algorithm of [9] and the modified quadratic penalty of [7] We also computed the PL estimates for the standard quadratic penalty function (which has nonuniform spatial resolution [7] Both algorithms enforced the nonnegativity constraint. Presented at 1997 IEEE Nuc. Sci. Symp. and Med. Im. Conf. For each of ....

J. A. Fessler and A. O. Hero, "Penalized maximumlikelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417--29, October 1995.

....images from Poisson sinogram measurements. The algorithm ffl monotonically increases the objective function, ffl is globally convergent, ffl naturally accommodates the nonnegativity constraint, ffl requires less CPU time per iteration than the spacealternating generalize EM (SAGE) algorithm [1]. For reconstruction problems (such as 2D PET and SPECT) where the system matrix G can be precomputed and stored, we recommend this new algorithm over our previously published algorithms for penalized likelihood reconstruction. For reconstruction problems where the system matrix is represented in ....

....PSfrag replacements hi (l) and q i (l; l n i ) Log likelihood h i (l) Surrogate q i (l; l n i ) l n i l Fig. 1 Illustration of 1D parabolic surrogate function. Note that q i (l; l n i ) h i (l) for l 0. III. THE NEW ALGORITHM We had previously recommended the SAGE algorithm [1] as a fast globally convergent algorithm for this problem. The construction of the SAGE algorithm requires certain minimizations that are somewhat unusual in the tomographic literature, and must be implemented carefully to achieve reasonable CPU time per iteration. The new algorithm we propose is ....

[Article contains additional citation context not shown here]

J. A. Fessler and A. O. Hero, "Penalized maximumlikelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Tr. Im. Proc., vol. 4, no. 10, pp. 1417--29, October 1995.

....accommodate nonnegativity constraints and nonquadratic convex penalties, and require a moderate number of exponentiations. The derivation of these transmission algorithms exploits two ideas underlying recent developments in algorithms for emission tomography: updating the parameters in groups [23, 24], and the convexity technique of De Pierro [25, 26] Integrating these two ideas leads to a new class of algorithms [27] that converge quickly and with less computation than previous statistical methods for transmission tomography. This work can be considered a generalization of previous methods ....

....these two ideas leads to a new class of algorithms [27] that converge quickly and with less computation than previous statistical methods for transmission tomography. This work can be considered a generalization of previous methods for tomographic image reconstruction based on sequential updates [28,5,10,29,23,24,11]. The fast convergence of sequential updates for tomographic problems was analyzed by Fourier methods and shown empirically to converge faster than simultaneous updates in [5] Tomographic reconstruction is an important case of the general problem of estimating superimposed signals [30 33] In ....

[Article contains additional citation context not shown here]

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, Oct. 1995.

....1 : J j ( 50; J k ( 50 0 : else 9 = 6) Notice that only pixels more than half inside the boundary included in the interior. The image that maximizes (4) can not be found analytically; therefore, an iterative method must be used. In the sequel we use the PML SAGE3 algorithm of [7] to perform the maximization. This EM type algorithm is characterized by monotonic, fast convergence. 4. LARGE fi LIMIT Next we give the limiting form of the PL reconstruction error as fi gets large. In this limiting case the penalty GammafiR( 5) dominates the objective (4) forcing to ....

J. A. Fessler and A.O. Hero. Penalized maximum likelihood image reconstruction using space alternating generalized em algorithms. IEEE Transactions on Image Processing, 4(10), October 1995.

....in the reconstruction . The smoothing parameter fi controls the tradeoff between the conflicting goals of maximizing ln f(y; and minimizing P ( The objective function maximum can not be found analytically; therefore, an iterative method must be used. In the sequel we use the SAGE3 algorithm [7] to maximize (3) This EM type algorithm is characterized by monotonic, fast convergence. Choosing a uniform weight scheme (e.g. all kj = 1) results in a global smoothing of the reconstructed image that in turn causes severe bias as image detail is blurred indiscriminantly across region ....

J. A. Fessler and A.O. Hero. Penalized maximum likelihood image reconstruction using space alternating generalized em algorithms. IEEE Transactions on Image Processing, 4(10), October 1995.

....Sadly, it is the golden oldies version of that EM algorithm [1] that is commercially available without regularization or acceleration. In fact, the imaging community parochially calls it the EM algorithm ) Our efforts to jazz up EM algorithms with penalty functions and faster convergence [2, 3] have yet to make the commercial hit parade. Within part of the medical imaging community, the continuing prevalence of classical EM over contemporary faster renditions is due to a lack of acceptance of methods for regularization. Without regularization, methods for accelerating EM algorithms for ....

....and the accompanying fast algorithms will see widespread use in medical imaging. In classical missing data problems in statistics, there is often exactly one natural choice for the augmented data, so the EM algorithm is appealing. In contrast, in the image reconstruction problem described in [3] and summarized in section 3.5, the augmented data has no physical interpretation. As noted in [3] one can derive the SAGE algorithm for image reconstruction from a non statistical perspective using only the concavity of the log likelihood and the convexity inequality. In some respects this ....

[Article contains additional citation context not shown here]

J A Fessler and A O Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, October 1995.

....accommodate nonnegativity constraints and nonquadratic convex penalties, and require a moderate number of exponentiations. The derivation of these transmission algorithms exploits two ideas underlying recent developments in algorithms for emission tomography: updating the parameters in groups [15, 16], and the convexity technique of De Pierro [17, 18] Integrating these two ideas leads to new algorithms that converge quickly with less computation than previous methods. II. Problem For brevity we consider the transmission measurement model without additive background events (random ....

.... optimization transfer idea [17, 18] and substitute a surrogate function OE( S ; n ) with a corresponding region of monotonicity RS IR pS that must satisfy: Phi( S ; n S ) Gamma Phi( n ) OE( S ; n ) Gamma OE( n S ; n ) 8 S 2 RS : 8) The SAGE like update [15, 16] then looks like: n 1 S = arg max S 2RS OE( S ; n ) 9) n 1 j = n j ; j 2 S: The condition (8) ensures immediately that the iterates produced by the above generic algorithm monotonically increase the objective: Phi( n 1 ) Phi( n ) We restrict attention here to ....

[Article contains additional citation context not shown here]

JA Fessler and AO Hero. Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms. IEEE Tr. Im. Proc., 4(10):1417--29, Oct. 1995.

....and a deconvolve shrink estimator. We maximized the nonquadratic penalized likelihood objective using the PMLSAGE algorithm, a variant of the iterative space alternating generalized expectation maximization (SAGE) algorithm of [20] adapted for penalized maximum likelihood image reconstruction [46]. We initialized PML SAGE with an unweighted penalized least squares estimate: A T A fi I) Gamma1 A 0 (Y Gamma r) which is linear so can be computed noniteratively. Here fi = fi P j A jk = P j A jk =Y j ) for k = 65 (cf [47] 48] By so initializing, only 30 iterations ....

J. A. Fessler and A. O. Hero, "Penalized maximum likelihood image reconstruction using space alternating generalized EM algorithms, " IEEE Trans. on Image Processing, vol. 3, , to appear Oct. 1995.

No context found.

J. Fessler and A. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, pp. 1417--1429, 1995.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithm," IEEE Trans. Imag. Process. 4, 10 (1995) 1417--1429.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417--1429, 1995.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417--1429, 1995.

No context found.

J. A. Fessler and A. O. Hero, "Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417--1429, 1995.

No context found.

J. A. Fessler and A. O. Hero, \Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms," IEEE Trans. Image Processing, vol. 4, no. 10, pp. 1417-1429, 1995.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC