OSA's Digital Library

Optics Express

Optics Express

  • Editor: Andrew M. Weiner
  • Vol. 21, Iss. 23 — Nov. 18, 2013
  • pp: 27668–27681
« Show journal navigation

Depth-variant deconvolution of 3D widefield fluorescence microscopy using the penalized maximum likelihood estimation method

Jeongtae Kim, Suhyeon An, Sohyun Ahn, and Boyoung Kim  »View Author Affiliations


Optics Express, Vol. 21, Issue 23, pp. 27668-27681 (2013)
http://dx.doi.org/10.1364/OE.21.027668


View Full Text Article

Acrobat PDF (802 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

We investigated the deconvolution of 3D widefield fluorescence microscopy using the penalized maximum likelihood estimation method and the depth-variant point spread function (DV-PSF). We build the DV-PSF by fitting a parameterized theoretical PSF model to an experimental microbead image. On the basis of the constructed DV-PSF, we restore the 3D widefield microscopy by minimizing an objective function consisting of a negative Poisson likelihood function and a total variation regularization function. In simulations and experiments, the proposed method showed better performance than existing methods.

© 2013 OSA

1. Introduction

Because a 3D widefield microscope requires significantly lower excitation light dosage than a 3D confocal microscope for acquiring satisfactory images, it is better suited for examining biological samples that can be easily bleached [1

1. J. G. McNally, T. Karpova, J. Cooper, and J. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999). [CrossRef] [PubMed]

]. However, one of the most serious disadvantages of the widefield microscope is that the acquired image is more blurred than that obtained in confocal microscopy [1

1. J. G. McNally, T. Karpova, J. Cooper, and J. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999). [CrossRef] [PubMed]

,2

2. P. Sarder and A. Nehorai, “Deconvolution methods for 3-D fluorescence microscopy images,” IEEE Signal Proc. Mag. 23, 32–45 (2006). [CrossRef]

]. The intensities of widefield microscopy are modeled using Poisson random variables, the mean values of which are the convolution of the true 3D image and the 3D depth-variant point spread function (DV-PSF) [3

3. C. Preza and J. A. Conchello, “Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 21, 1593–1601 (2004). [CrossRef]

5

5. J. G. McNally, C. Preza, J. A. Conchello, and L. J. Thomas, “Artifacts in computational optical-sectioning microscopy,” J. Opt. Soc. Am. A 11, 1056–1067 (1994). [CrossRef]

]. Therefore, image deconvolution is crucial for achieving enhanced 3D widefield microscopy. Such restored microscopy, often called deconvolution microscopy or computational optical sectioning microscopy (COSM), has been studied extensively [1

1. J. G. McNally, T. Karpova, J. Cooper, and J. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999). [CrossRef] [PubMed]

12

12. P. J. Verveer and T. M. Jovin, “Efficient superresolution restoration algorithms using maximum a posteriori estimations with application to fluorescence microscopy,” J. Opt. Soc. Am. A 14, 1696–1706 (1997). [CrossRef]

]. However, despite huge efforts by many researchers, the performance of the deconvolution is not yet satisfactory since the deconvolution is very challenging because the PSF is unknown as well as depth variant [13

13. S. F. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A 8, 1601–1613 (1991). [CrossRef]

, 14

14. F. Aguet, D. Van De Ville, and M. Unser, “An accurate PSF model with few parameters for axially shift-variant deconvolution,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2008), 157–160.

]. The PSF for 3D microscopy is depth variant because the depth-variant optical path difference (OPD) between the ideal and actual imaging conditions depends on the depth of the object location [13

13. S. F. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A 8, 1601–1613 (1991). [CrossRef]

, 14

14. F. Aguet, D. Van De Ville, and M. Unser, “An accurate PSF model with few parameters for axially shift-variant deconvolution,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2008), 157–160.

].

Despite true PSF being depth variant, in many existing methods, one assumes a depth-invariant PSF for simplicity and applies well-developed shift-invariant deconvolution methods such as the Richardson–Lucy (RL) method [7

7. J. Markham and J. A. Conchello, “Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur,” J. Opt. Soc. Am. A 16, 2377–2391 (1999). [CrossRef]

11

11. P. Pankajakshan, B. Zhang, L. Blanc-Féraud, Z. Kam, J. Olivo-Marin, and J. Zerubia, “Blind deconvolution for diffraction-limited fluorescence microscopy,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2008), 740–743.

]. Since the PSF is unknown, in some methods, one first estimates the PSF by using a microbead image and uses the estimated PSF for deconvolution. Other methods attempt to estimate both the unknown PSF and the 3D image simultaneously by using blind deconvolution methods [7

7. J. Markham and J. A. Conchello, “Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur,” J. Opt. Soc. Am. A 16, 2377–2391 (1999). [CrossRef]

, 11

11. P. Pankajakshan, B. Zhang, L. Blanc-Féraud, Z. Kam, J. Olivo-Marin, and J. Zerubia, “Blind deconvolution for diffraction-limited fluorescence microscopy,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2008), 740–743.

]. The performance of such depth-invariant restoration methods is limited because the PSF used for the deconvolution is not accurate, especially for thick samples in the depth direction [13

13. S. F. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A 8, 1601–1613 (1991). [CrossRef]

]. To overcome this limitation, two investigators attempted to use a DV-PSF for deconvolution, including an investigation of the maximum likelihood estimation (MLE) of the true object using an analytically computed [3

3. C. Preza and J. A. Conchello, “Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 21, 1593–1601 (2004). [CrossRef]

] or experimentally estimated DV-PSF [4

4. J. Shaevitz and D. Fletcher, “Enhanced three-dimensional deconvolution microscopy using a measured depth-varying point-spread function,” J. Opt. Soc. Am. A 24, 2622–2627 (2007). [CrossRef]

]. The former method is based on a parametric PSF model using the Kirchhoff diffraction integral formula [13

13. S. F. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A 8, 1601–1613 (1991). [CrossRef]

], whereas the latter method utilizes optical trapping fluorescence microscopy for determining the PSF [4

4. J. Shaevitz and D. Fletcher, “Enhanced three-dimensional deconvolution microscopy using a measured depth-varying point-spread function,” J. Opt. Soc. Am. A 24, 2622–2627 (2007). [CrossRef]

]. To our knowledge, there have been no investigations on blind deconvolution based on the DV-PSF except for one recent investigation for confocal microscopy [15

15. S. Ben Hadj, G. Blanc-Feraud, G. Aubert, and Engler, “Blind restoration of confocal microscopy images in presence of a depth-variant blur and poisson noise,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2013), 915–919.

], probably because of the demanding computations involved. The recent investigation utilized a fast optimization method to solve the blind deconvolution problem [16

16. S. Bonettini, R. Zanella, and L. Zanni, “A scaled gradient projection method for constrained image deblurring,” Inverse Probl. 25, 015002 (2009). [CrossRef]

, 17

17. R. Zanella, G. Zanghirati, R. Cavicchioli, L. Zanni, P. Boccacci, M. Bertero, and G. Vicidomini, “Towards real-time image deconvolution: application to confocal and STED microscopy,” Sci. Rep. 3, 2523(2013). [CrossRef] [PubMed]

]. Because the intensity at each location in acquired microscopy is modeled by a Poisson random variable [3

3. C. Preza and J. A. Conchello, “Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 21, 1593–1601 (2004). [CrossRef]

, 8

8. J. Markham and J. A. Conchello, “Fast maximum-likelihood image-restoration algorithms for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 18, 1062–1071 (2001). [CrossRef]

], most existing deconvolution methods for 3D fluorescence microscopy (either depth invariant or depth variant) are based on the RL iteration [18

18. W. H. Richardson, “Bayesian-based iterative method of image restoration,” J. Opt. Soc. Am. 62, 55–59 (1972). [CrossRef]

] that maximizes the Poisson likelihood function using the expectation maximization (EM) optimization method. In that sense, it is not surprising that the MLE using a DV-PSF yields a depth-variant RL (DV-RL) iteration that is similar to the RL iteration except for the convolution during the iteration being depth variant [3

3. C. Preza and J. A. Conchello, “Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 21, 1593–1601 (2004). [CrossRef]

]. Although the DV-RL method is more accurate than the depth-invariant methods, its performance is still limited because of its noise sensitivity. Because the maximization of the likelihood function is ill-posed in the sense that the solution is not unique, MLE may yield a noisy image [2

2. P. Sarder and A. Nehorai, “Deconvolution methods for 3-D fluorescence microscopy images,” IEEE Signal Proc. Mag. 23, 32–45 (2006). [CrossRef]

, 19

19. J. Fessler, “Image reconstruction: Algorithms and analysis,” Online preprint of book in preparation.

]. Therefore, one must incorporate a regularization function based on a priori information on the true image to yield a more enhanced image [8

8. J. Markham and J. A. Conchello, “Fast maximum-likelihood image-restoration algorithms for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 18, 1062–1071 (2001). [CrossRef]

10

10. J. A. Conchello and J. G. McNally, “Fast regularization technique for expectation maximization algorithm for computational optical sectioning microscopy,” Proc. SPIE 2655, 199–208 (1996) [CrossRef]

].

Regularized MLE methods using a DV-PSF (equivalently, depth-variant maximum a posteriori estimation) have been seldom studied except for recent two investigations for confocal microscopy [20

20. S. Ben Hadj, L. Blanc-Feraud, E. Maalouf, B. Colicchio, and A. Dieterlen, “Depth-variant image restoration in 3D fluorescence microscopy: Two approaches under gaussian and poissonian noise conditions,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2012), 1671–1674.

, 21

21. S. Ben Hadj and L. Blanc-Feraud, “Modeling and removing depth variant blur in 3D fluorescence microscopy,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2012), 689–692.

]. On the contrary, there exist several investigations on regularized depth-invariant deconvolution methods [6

6. S. Joshi and M. I. Miller, “Maximum a posteriori estimation with Good’s roughness for three-dimensional optical-sectioning microscopy,” J. Opt. Soc. Am. A 10, 1078–1085 (1993). [CrossRef] [PubMed]

,8

8. J. Markham and J. A. Conchello, “Fast maximum-likelihood image-restoration algorithms for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 18, 1062–1071 (2001). [CrossRef]

10

10. J. A. Conchello and J. G. McNally, “Fast regularization technique for expectation maximization algorithm for computational optical sectioning microscopy,” Proc. SPIE 2655, 199–208 (1996) [CrossRef]

,12

12. P. J. Verveer and T. M. Jovin, “Efficient superresolution restoration algorithms using maximum a posteriori estimations with application to fluorescence microscopy,” J. Opt. Soc. Am. A 14, 1696–1706 (1997). [CrossRef]

,22

22. N. Dey, L. Blanc-Feraud, C. Zimmer, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “A deconvolution method for confocal microscopy with total variation regularization,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2004), 1223–1226

]. The performance of depth-invariant methods is not satisfactory due to incorrect PSF model. In addition, the performance of many existing methods might be limited since ineffective regularization functions were used. It is well known that the most useful a priori information is that of a true object with small intensity variations in the spatial domain, which requires a coupled regularization function defined by intensity differences in adjacent pixels. However, probably for simplicity, existing methods are based either on the intensity of each pixel (not the difference between pixels) or on approximations for computing differences between neighboring pixels [20

20. S. Ben Hadj, L. Blanc-Feraud, E. Maalouf, B. Colicchio, and A. Dieterlen, “Depth-variant image restoration in 3D fluorescence microscopy: Two approaches under gaussian and poissonian noise conditions,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2012), 1671–1674.

,22

22. N. Dey, L. Blanc-Feraud, C. Zimmer, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “A deconvolution method for confocal microscopy with total variation regularization,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2004), 1223–1226

,23

23. P. J. Green, “On use of the EM for penalized likelihood estimation,” J. R. Stat. Soc. B 52, 443–452 (1990).

]. We suppose that the scarcity of coupled regularization functions can be attributed to the belief that performing an RL-type pixelwise update is impossible for a coupled regularization function [10

10. J. A. Conchello and J. G. McNally, “Fast regularization technique for expectation maximization algorithm for computational optical sectioning microscopy,” Proc. SPIE 2655, 199–208 (1996) [CrossRef]

]. Although there are investigations based on the coupled roughness penalty function [6

6. S. Joshi and M. I. Miller, “Maximum a posteriori estimation with Good’s roughness for three-dimensional optical-sectioning microscopy,” J. Opt. Soc. Am. A 10, 1078–1085 (1993). [CrossRef] [PubMed]

,21

21. S. Ben Hadj and L. Blanc-Feraud, “Modeling and removing depth variant blur in 3D fluorescence microscopy,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2012), 689–692.

], the methods require the simultaneous optimization of entire pixels, which is often computationally intractable. Existing regularized deconvolution methods with DV-PSF do not provide RL type iterative update algorithm to minimize negative Poisson likelihood plus regularization function. One method is based on approximations to compute regularization function [20

20. S. Ben Hadj, L. Blanc-Feraud, E. Maalouf, B. Colicchio, and A. Dieterlen, “Depth-variant image restoration in 3D fluorescence microscopy: Two approaches under gaussian and poissonian noise conditions,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2012), 1671–1674.

] and the other method is based on Gaussian likelihood function, which is an approximation of Poisson likelihood function [21

21. S. Ben Hadj and L. Blanc-Feraud, “Modeling and removing depth variant blur in 3D fluorescence microscopy,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2012), 689–692.

].

To overcome the limitations of existing methods, we investigate a regularized MLE method using a DV-PSF. For the regularization function, we adopt the well-known total variation penalty function that can preserve edges better than the quadratic roughness penalty function. To compute the depth-variant penalized (i.e., regularized) MLE, we apply the generalized EM method, which yields a pixelwise iteration that converges to the minimizer of the penalized negative likelihood function. The depth-variant generalized EM (DV-GEM) method is based on the minimization of a separable quadratic surrogate function for the penalized negative Poisson likelihood function, which was proven to be effective in medical image reconstruction problems [24

24. A. De Pierro, “A modified expectation maximization algorithm for penalized likelihood estimation in emission tomography,” IEEE Trans. Med. Imag. 14, 132–137 (1995). [CrossRef]

26

26. J. H. Chang, J. Anderson, and J. Votaw, “Regularized image reconstruction algorithms for positron emission tomography,” IEEE Trans. Med. Imag. 23, 1165–1175 (2004). [CrossRef]

]. We estimate the DV-PSF by fitting a simplified parametric DV-PSF model [14

14. F. Aguet, D. Van De Ville, and M. Unser, “An accurate PSF model with few parameters for axially shift-variant deconvolution,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2008), 157–160.

] to a small-size microbead image. We prove the effectiveness of the DV-GEM method in comparison with conventional methods such as the depth-invariant RL (INV-RL) method, the DV-RL method, and the depth-invariant GEM (INV-GEM) method in simulations and experiments using microsphere beads (Polyscience Inc., PA, USA) and an Olympus IX-71 inverted 3D wide-field microscope.

2. Problem formulation

2.1. Image and PSF model

2.2. Image restoration

Based on the image acquisition model in Eq. (1), the goal of 3D fluorescence microscopy restoration is the accurate estimation of the true object s = [s(x0), s(x1),..., s(xN−1)] using g = [g(x0), g(x1),..., g(xN−1)]. One of the most natural approaches for the restoration entails the determination of the image that maximizes the Poisson likelihood function using a constructed PSF (either depth invariant or depth variant). The maximization using a depth-invariant PSF can be solved by applying an EM algorithm, which leads to an INV-RL iteration [18

18. W. H. Richardson, “Bayesian-based iterative method of image restoration,” J. Opt. Soc. Am. 62, 55–59 (1972). [CrossRef]

, 27

27. L. A. Shepp and Y. Vardi, “Maximum likelihood reconstruction for emission tomography,” IEEE Trans. Med. Imag. 1, 113–122 (1982). [CrossRef]

]. The RL iteration has the advantage of monotonic convergence. In addition, it does not require the inversion of the huge Hessian matrix that is often required in general optimization methods [28

28. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C++ - The Art of Scientific Computing (Cambridge University, 2002).

]. In many methods, the INV-RL iteration was applied under the assumption that the PSF does not change within a sample. However, the performance of such methods is not satisfactory, especially for thick samples in which the true PSF changes significantly according to the change of depth [13

13. S. F. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A 8, 1601–1613 (1991). [CrossRef]

].

3. Proposed method

3.1. Construction of a depth-variant PSF

We first construct a DV-PSF using Eq. (2) and Eq. (3). For this, we acquire the microscopy of a very small size microbead and assume that the microscopy is the PSF at the location of the microbead. Because we do not have knowledge about the true location of the microbead, the MLE of the unknown parameters are the joint estimation of the parameters and unknown location obtained by maximizing the Poisson likelihood function defined as follows:
(θ^,x^o)=argmaxθ,xoL(θ,xo;g),
(12)
where
L(θ,xo;g)=ih(xixo,zo;θ)+g(xi)log(h(xixo)).
(13)
It is to be noted that we assume s(xo) = δ(xxo) for computing the likelihood function. One can find the optimal parameters and the location of the microbead using a nonlinear optimization method.

3.2. Depth-variant penalized maximum likelihood estimation

4. Results

4.1. PSF estimation

We prepared a sample of a diluted solution of 0.2-μm-diameter Fluoresbrite Yellow Green Fluorescent Microspheres (Polyscience Inc., PA, USA) and dried the solution onto a cover slip. Then, the sample was imaged using a 40×, 0.95 NA objective lens with an Olympus IX-71 inverted microscope. Using the 65 × 65 × 59 acquired image with spacings of 0.11 × 0.11 × 0.64 μm, we estimated the PSF parameters and the location of the microbead using the method defined in Eq. (12). To estimate the parameters that minimize the objective function defined in Eq. (12), we used the simplex optimization method [28

28. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C++ - The Art of Scientific Computing (Cambridge University, 2002).

]. Figures 1(a) and 1(b) show the xz sections of the acquired microbead image and the estimated PSF image (the estimated center location of the microbead on the z axis is 26.9 μm), respectively, whereas Figs. 1(c) and 1(d) show the xy sections of the acquired bead and the estimated PSF image (a slice at z = 36.7 μm), respectively. The estimated ni and ns values are 1.53 and 1.13, respectively. The normalized correlation coefficient (COR) value of the acquired image and the estimated PSF image was 0.903.

Fig. 1 Microbead image and fitted PSF: (a) xz profile of acquired microbead image; (b) xz profile of estimated PSF image; (c) xy profile of acquired image at z = 36.7 μm; (d) xy profile of estimated PSF image at z = 36.7 μm.

4.2. Simulations

We conducted simulation studies to evaluate the performance of the proposed method in comparison with the INV-RL method, the INV-GEM method, and the DV-RL method. First, we generated a synthetic 20-μm-diameter microsphere object with its center location on the z axis being at a depth of 10 μm. Then, we convolved the synthetic object using the constructed PSF model under Poisson noise. We generated such synthetic noisy and blurred observed images (65 × 65 × 111, with spacings of 0.5 μm for the x, y, and z axes) for five different signal to noise ratios (SNRs) and attempted to restore the true object using the INV-RL, INV-GEM, DV-RL, and DV-GEM methods. We repeated iterations of the four methods for 200 times, which ensures the relative change of objective function value is less than 1 × e−5 for all the methods. We also tested the performance of the INV-RL and the DV-RL method with a stopping strategy [32

32. J. Llacer and E. Veklerov, “Feasible images and practical stopping rules for iterative algorithms in emission tomography,” IEEE Trans. Med. Imag. 8, 186–193 (1989). [CrossRef]

], which attempts to avoid restoring noisy image by early termination of the iteration before convergence. To do that, we compute mean square error between restored image and true image, and terminate the iteration when the error is minimized. Although this strategy is not applicable for real experimental data since true image is not available, we report this results for comparison purpose.

Figures 2(a) and 2(b) show the xz profile of the true microsphere-shape synthetic object and that of the noisy blurred image used for simulation. Figures 2(c)–2(h) show the restored images using the INV-RL, INV-RL with stopping strategy, INV-GEM, DV-RL, DV-RL with stopping strategy, and DV-GEM methods. For depth-invariant methods such as INV-RL and INV-GEM, we used the PSF model at z = 10 μm for deconvolution. Figures 3(a) and 3(b) show the xy section (at z = 10 μm) of the true object and the noisy blurred image. Figures 3(c)–3(h) show the restored images using the INV-RL, INV-RL with stopping strategy, INV-GEM, DV-RL, DV-RL with stopping strategy, and DV-GEM methods, respectively. As one can see in the figures, invariant methods such as INV-RL and INV-GEM were unable to effectively restore the xz section of the true object. Although the DV-RL method was able to restore the spherical shape in the xz profile, the image was noisy because the method does not incorporate a regularization function. Although the stopping strategy was able to reduce noise a little bit more than the INV-RL and DV-RL methods without stopping strategy, the resulting images (Figs. 2(d), 2(g), 3(d) and 3(g)) are still noisy. Compared with other methods, the DV-GEM method restored the true object more effectively even in the presence of noise.

Fig. 2 Images of xz profiles in simulation: (a) true image; (b) blurred and noisy image (SNR=15dB); (c) INV-RL; (d) INV-RL (stopping) (e) INV-GEM; (f) DV-RL; (g) DV-RL (stopping); (f) DV-GEM.
Fig. 3 Images of xy profiles in simulation: (a) true image; (b) blurred and noisy image (SNR=15dB); (c) INV-RL; (d) INV-RL (stopping) (e) INV-GEM; (f) DV-RL; (g) DV-RL (stopping); (f) DV-GEM.

To study the statistical properties of the INV-RL, INV-GEM, DV-DL and DV-GEM methods, we repeated the restoration 10 times using 10 different noise realizations. Table 1 shows the average COR values between the true object and the restored images for the 10 restorations obtained by using the four methods. As shown in the table, the proposed DV-GEM method outperformed the other methods. Note that the performance of the depth-variant methods was better than that of the depth-invariant methods. In addition, the DV-GEM method outperformed the DV-RL method for all SNR cases. In particular, if the noise is severe, the performance of the DV-GEM method was much better than that of the DV-RL method. When the SNR was very good (e.g., 30 dB case), the performance of the DV-RL was almost the same as that of the DV-GEM method. For the INV-GEM method and the DV-GEM method, we selected the regularization parameter manually in a way such that restored images are smooth enough. One may attempt to automatically tune the regularization parameter using methods such as generalized cross validation [33

33. N. Nguyen, P. Milanfar, and G. Golub, “Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement,” IEEE Trans. Image Process. 10, 1299–1308 (2001). [CrossRef]

]. We defer investigation on the automatic regularization parameter tuning to a future study.

Table 1. COR values of true object image and restored images obtained by using the four methods.

table-icon
View This Table
| View All Tables

We also report computation time of the four methods implemented using MATLAB (Mathworks, USA). Table 2 shows computation time of the four methods per iteration on a workstation that has two Intel Xeon X5650 processors (2.67GHz) and 96 GB memory. As expected, depth-variant methods required much more computation time due to computationally intensive depth-variant convolution operation. However, the DV-GEM method required only slightly more computation time than the DV-RL method. Note that computation time for depth-variant methods can be reduced using a parallel processing algorithm and/or more advanced computers.

Table 2. Computation time of the four methods per iteration (unit is second).

table-icon
View This Table
| View All Tables

4.3. Experiments

Fig. 4 Images of xz profiles in real experiments: (a) observed; (b) INV-RL; (c) INV-GEM; (d) DV-RL; (e) DV-GEM.
Fig. 5 Images of xy profiles at z = 17.97 μm in real experiments: (a) observed; (b) INV-RL; (c) INV-GEM; (d) DV-RL; (e) DV-GEM.

5. Conclusions

We investigated the deconvolution of 3D widefield fluorescence microscopy images using the penalized MLE method and the DV-PSF. Using an estimated DV-PSF by fitting a parameterized theoretical PSF model to an experimental microbead image, we applied the depth-variant penalized maximum likelihood method for the deconvolution of 3D widefield microscopy images. For minimizing the penalized negative Poisson likelihood function with the TV penalty function, we applied the generalized EM method. In simulations and real experiments, the proposed method showed better performance than the depth-invariant RL method, the depth-invariant penalized maximum likelihood method, and the depth-variant RL method. We believe that the proposed method will be very useful for deconvolution microscopy.

Acknowledgments

This work was supported by the Korean Science Engineering Foundation ( KOSEF R17-2008-041-01001-0) funded by the Korean Government and Ewha Global Top 5 Grant 2011 of Ewha Womans University.

References and links

1.

J. G. McNally, T. Karpova, J. Cooper, and J. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19, 373–385 (1999). [CrossRef] [PubMed]

2.

P. Sarder and A. Nehorai, “Deconvolution methods for 3-D fluorescence microscopy images,” IEEE Signal Proc. Mag. 23, 32–45 (2006). [CrossRef]

3.

C. Preza and J. A. Conchello, “Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 21, 1593–1601 (2004). [CrossRef]

4.

J. Shaevitz and D. Fletcher, “Enhanced three-dimensional deconvolution microscopy using a measured depth-varying point-spread function,” J. Opt. Soc. Am. A 24, 2622–2627 (2007). [CrossRef]

5.

J. G. McNally, C. Preza, J. A. Conchello, and L. J. Thomas, “Artifacts in computational optical-sectioning microscopy,” J. Opt. Soc. Am. A 11, 1056–1067 (1994). [CrossRef]

6.

S. Joshi and M. I. Miller, “Maximum a posteriori estimation with Good’s roughness for three-dimensional optical-sectioning microscopy,” J. Opt. Soc. Am. A 10, 1078–1085 (1993). [CrossRef] [PubMed]

7.

J. Markham and J. A. Conchello, “Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur,” J. Opt. Soc. Am. A 16, 2377–2391 (1999). [CrossRef]

8.

J. Markham and J. A. Conchello, “Fast maximum-likelihood image-restoration algorithms for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 18, 1062–1071 (2001). [CrossRef]

9.

C. Preza, M. I. Miller, J. Lewis, J. Thomas, and J. G. McNally, “Regularized linear method for reconstruction of three-dimensional microscopic objects from optical sections,” J. Opt. Soc. Am. A 9, 219–228 (1992). [CrossRef] [PubMed]

10.

J. A. Conchello and J. G. McNally, “Fast regularization technique for expectation maximization algorithm for computational optical sectioning microscopy,” Proc. SPIE 2655, 199–208 (1996) [CrossRef]

11.

P. Pankajakshan, B. Zhang, L. Blanc-Féraud, Z. Kam, J. Olivo-Marin, and J. Zerubia, “Blind deconvolution for diffraction-limited fluorescence microscopy,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2008), 740–743.

12.

P. J. Verveer and T. M. Jovin, “Efficient superresolution restoration algorithms using maximum a posteriori estimations with application to fluorescence microscopy,” J. Opt. Soc. Am. A 14, 1696–1706 (1997). [CrossRef]

13.

S. F. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A 8, 1601–1613 (1991). [CrossRef]

14.

F. Aguet, D. Van De Ville, and M. Unser, “An accurate PSF model with few parameters for axially shift-variant deconvolution,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2008), 157–160.

15.

S. Ben Hadj, G. Blanc-Feraud, G. Aubert, and Engler, “Blind restoration of confocal microscopy images in presence of a depth-variant blur and poisson noise,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2013), 915–919.

16.

S. Bonettini, R. Zanella, and L. Zanni, “A scaled gradient projection method for constrained image deblurring,” Inverse Probl. 25, 015002 (2009). [CrossRef]

17.

R. Zanella, G. Zanghirati, R. Cavicchioli, L. Zanni, P. Boccacci, M. Bertero, and G. Vicidomini, “Towards real-time image deconvolution: application to confocal and STED microscopy,” Sci. Rep. 3, 2523(2013). [CrossRef] [PubMed]

18.

W. H. Richardson, “Bayesian-based iterative method of image restoration,” J. Opt. Soc. Am. 62, 55–59 (1972). [CrossRef]

19.

J. Fessler, “Image reconstruction: Algorithms and analysis,” Online preprint of book in preparation.

20.

S. Ben Hadj, L. Blanc-Feraud, E. Maalouf, B. Colicchio, and A. Dieterlen, “Depth-variant image restoration in 3D fluorescence microscopy: Two approaches under gaussian and poissonian noise conditions,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2012), 1671–1674.

21.

S. Ben Hadj and L. Blanc-Feraud, “Modeling and removing depth variant blur in 3D fluorescence microscopy,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2012), 689–692.

22.

N. Dey, L. Blanc-Feraud, C. Zimmer, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “A deconvolution method for confocal microscopy with total variation regularization,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2004), 1223–1226

23.

P. J. Green, “On use of the EM for penalized likelihood estimation,” J. R. Stat. Soc. B 52, 443–452 (1990).

24.

A. De Pierro, “A modified expectation maximization algorithm for penalized likelihood estimation in emission tomography,” IEEE Trans. Med. Imag. 14, 132–137 (1995). [CrossRef]

25.

J. Fessler and A. Hero, “Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms,” IEEE Trans. Image Process. 4, 1417–1429 (1995). [CrossRef] [PubMed]

26.

J. H. Chang, J. Anderson, and J. Votaw, “Regularized image reconstruction algorithms for positron emission tomography,” IEEE Trans. Med. Imag. 23, 1165–1175 (2004). [CrossRef]

27.

L. A. Shepp and Y. Vardi, “Maximum likelihood reconstruction for emission tomography,” IEEE Trans. Med. Imag. 1, 113–122 (1982). [CrossRef]

28.

W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C++ - The Art of Scientific Computing (Cambridge University, 2002).

29.

J. F. Aujol, “Some first-order algorithms for total variation based image restoration,” J. Math. Imaging Vis. 34, 307–327 (2009). [CrossRef]

30.

S. Bonettini and V. Ruggiero, “An alternating extragradient method for total variation-based image restoration from poisson data,” Inverse Probl. 27, 095001 (2011). [CrossRef]

31.

P. Huber, Robust Statistics (Wiley, 1974).

32.

J. Llacer and E. Veklerov, “Feasible images and practical stopping rules for iterative algorithms in emission tomography,” IEEE Trans. Med. Imag. 8, 186–193 (1989). [CrossRef]

33.

N. Nguyen, P. Milanfar, and G. Golub, “Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement,” IEEE Trans. Image Process. 10, 1299–1308 (2001). [CrossRef]

34.

F. Aguet, D. V. D. Ville, and M. Unser, “A maximum-likelihood formalism for sub-resolution axial localization of fluorescent nanoparticles,” Opt. Express 13, 10503–10522 (2005). [CrossRef] [PubMed]

OCIS Codes
(100.3020) Image processing : Image reconstruction-restoration
(100.3190) Image processing : Inverse problems
(180.2520) Microscopy : Fluorescence microscopy

ToC Category:
Image Processing

History
Original Manuscript: September 6, 2013
Revised Manuscript: October 22, 2013
Manuscript Accepted: October 27, 2013
Published: November 4, 2013

Virtual Issues
Vol. 9, Iss. 1 Virtual Journal for Biomedical Optics

Citation
Jeongtae Kim, Suhyeon An, Sohyun Ahn, and Boyoung Kim, "Depth-variant deconvolution of 3D widefield fluorescence microscopy using the penalized maximum likelihood estimation method," Opt. Express 21, 27668-27681 (2013)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-21-23-27668


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. J. G. McNally, T. Karpova, J. Cooper, and J. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods19, 373–385 (1999). [CrossRef] [PubMed]
  2. P. Sarder and A. Nehorai, “Deconvolution methods for 3-D fluorescence microscopy images,” IEEE Signal Proc. Mag.23, 32–45 (2006). [CrossRef]
  3. C. Preza and J. A. Conchello, “Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A21, 1593–1601 (2004). [CrossRef]
  4. J. Shaevitz and D. Fletcher, “Enhanced three-dimensional deconvolution microscopy using a measured depth-varying point-spread function,” J. Opt. Soc. Am. A24, 2622–2627 (2007). [CrossRef]
  5. J. G. McNally, C. Preza, J. A. Conchello, and L. J. Thomas, “Artifacts in computational optical-sectioning microscopy,” J. Opt. Soc. Am. A11, 1056–1067 (1994). [CrossRef]
  6. S. Joshi and M. I. Miller, “Maximum a posteriori estimation with Good’s roughness for three-dimensional optical-sectioning microscopy,” J. Opt. Soc. Am. A10, 1078–1085 (1993). [CrossRef] [PubMed]
  7. J. Markham and J. A. Conchello, “Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur,” J. Opt. Soc. Am. A16, 2377–2391 (1999). [CrossRef]
  8. J. Markham and J. A. Conchello, “Fast maximum-likelihood image-restoration algorithms for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A18, 1062–1071 (2001). [CrossRef]
  9. C. Preza, M. I. Miller, J. Lewis, J. Thomas, and J. G. McNally, “Regularized linear method for reconstruction of three-dimensional microscopic objects from optical sections,” J. Opt. Soc. Am. A9, 219–228 (1992). [CrossRef] [PubMed]
  10. J. A. Conchello and J. G. McNally, “Fast regularization technique for expectation maximization algorithm for computational optical sectioning microscopy,” Proc. SPIE2655, 199–208 (1996) [CrossRef]
  11. P. Pankajakshan, B. Zhang, L. Blanc-Féraud, Z. Kam, J. Olivo-Marin, and J. Zerubia, “Blind deconvolution for diffraction-limited fluorescence microscopy,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2008), 740–743.
  12. P. J. Verveer and T. M. Jovin, “Efficient superresolution restoration algorithms using maximum a posteriori estimations with application to fluorescence microscopy,” J. Opt. Soc. Am. A14, 1696–1706 (1997). [CrossRef]
  13. S. F. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A8, 1601–1613 (1991). [CrossRef]
  14. F. Aguet, D. Van De Ville, and M. Unser, “An accurate PSF model with few parameters for axially shift-variant deconvolution,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2008), 157–160.
  15. S. Ben Hadj, G. Blanc-Feraud, G. Aubert, and Engler, “Blind restoration of confocal microscopy images in presence of a depth-variant blur and poisson noise,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2013), 915–919.
  16. S. Bonettini, R. Zanella, and L. Zanni, “A scaled gradient projection method for constrained image deblurring,” Inverse Probl.25, 015002 (2009). [CrossRef]
  17. R. Zanella, G. Zanghirati, R. Cavicchioli, L. Zanni, P. Boccacci, M. Bertero, and G. Vicidomini, “Towards real-time image deconvolution: application to confocal and STED microscopy,” Sci. Rep.3, 2523(2013). [CrossRef] [PubMed]
  18. W. H. Richardson, “Bayesian-based iterative method of image restoration,” J. Opt. Soc. Am.62, 55–59 (1972). [CrossRef]
  19. J. Fessler, “Image reconstruction: Algorithms and analysis,” Online preprint of book in preparation.
  20. S. Ben Hadj, L. Blanc-Feraud, E. Maalouf, B. Colicchio, and A. Dieterlen, “Depth-variant image restoration in 3D fluorescence microscopy: Two approaches under gaussian and poissonian noise conditions,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2012), 1671–1674.
  21. S. Ben Hadj and L. Blanc-Feraud, “Modeling and removing depth variant blur in 3D fluorescence microscopy,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2012), 689–692.
  22. N. Dey, L. Blanc-Feraud, C. Zimmer, Z. Kam, J. C. Olivo-Marin, and J. Zerubia, “A deconvolution method for confocal microscopy with total variation regularization,” in Proceedings of IEEE International Symposium on Biomedical Imaging (IEEE, 2004), 1223–1226
  23. P. J. Green, “On use of the EM for penalized likelihood estimation,” J. R. Stat. Soc. B52, 443–452 (1990).
  24. A. De Pierro, “A modified expectation maximization algorithm for penalized likelihood estimation in emission tomography,” IEEE Trans. Med. Imag.14, 132–137 (1995). [CrossRef]
  25. J. Fessler and A. Hero, “Penalized maximum-likelihood image reconstruction using space-alternating generalized EM algorithms,” IEEE Trans. Image Process.4, 1417–1429 (1995). [CrossRef] [PubMed]
  26. J. H. Chang, J. Anderson, and J. Votaw, “Regularized image reconstruction algorithms for positron emission tomography,” IEEE Trans. Med. Imag.23, 1165–1175 (2004). [CrossRef]
  27. L. A. Shepp and Y. Vardi, “Maximum likelihood reconstruction for emission tomography,” IEEE Trans. Med. Imag.1, 113–122 (1982). [CrossRef]
  28. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C++ - The Art of Scientific Computing (Cambridge University, 2002).
  29. J. F. Aujol, “Some first-order algorithms for total variation based image restoration,” J. Math. Imaging Vis.34, 307–327 (2009). [CrossRef]
  30. S. Bonettini and V. Ruggiero, “An alternating extragradient method for total variation-based image restoration from poisson data,” Inverse Probl.27, 095001 (2011). [CrossRef]
  31. P. Huber, Robust Statistics (Wiley, 1974).
  32. J. Llacer and E. Veklerov, “Feasible images and practical stopping rules for iterative algorithms in emission tomography,” IEEE Trans. Med. Imag.8, 186–193 (1989). [CrossRef]
  33. N. Nguyen, P. Milanfar, and G. Golub, “Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement,” IEEE Trans. Image Process.10, 1299–1308 (2001). [CrossRef]
  34. F. Aguet, D. V. D. Ville, and M. Unser, “A maximum-likelihood formalism for sub-resolution axial localization of fluorescent nanoparticles,” Opt. Express13, 10503–10522 (2005). [CrossRef] [PubMed]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Figures

Fig. 1 Fig. 2 Fig. 3
 
Fig. 4 Fig. 5
 

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited