## Non-iterative determination of pattern phase in structured illumination microscopy using auto-correlations in Fourier space |

Optics Express, Vol. 21, Issue 21, pp. 24692-24701 (2013)

http://dx.doi.org/10.1364/OE.21.024692

Acrobat PDF (1461 KB)

### Abstract

The artefact-free reconstruction of structured illumination microscopy images requires precise knowledge of the pattern phases in the raw images. If this parameter cannot be controlled precisely enough in an experimental setup, the phases have to be determined *a posteriori* from the acquired data. While an iterative optimisation based on cross-correlations between individual Fourier images yields accurate results, it is rather time-consuming. Here I present a fast non-iterative technique which determines each pattern phase from an auto-correlation of the respective Fourier image. In addition to improving the speed of the reconstruction, simulations show that this method is also more robust, yielding errors of typically less than *λ*/500 under realistic signal-to-noise levels.

© 2013 OSA

## 1. Introduction

1. R. Heintzmann and C. Cremer, “Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating,” Proc. SPIE **3568**, 185–196 (1999). [CrossRef]

*a posteriori*from the acquired data. Shroff

*et al*. showed that the pattern phase of each individual raw image can be determined from the phase of the peak at the pattern frequency in Fourier space [4

4. S. A. Shroff, J. R. Fienup, and D. R. Williams, “Phase-shift estimation in sinusoidally illuminated images for lateral superresolution,” JOSA A **26**, 413–424 (2009). [CrossRef] [PubMed]

5. K. Wicker, O. Mandula, G. Best, R. Fiolka, and R. Heintzmann, “Phase optimisation for structured illumination microscopy,” Opt. Express **21**, 2032–2049 (2013). [CrossRef] [PubMed]

*et al*.

5. K. Wicker, O. Mandula, G. Best, R. Fiolka, and R. Heintzmann, “Phase optimisation for structured illumination microscopy,” Opt. Express **21**, 2032–2049 (2013). [CrossRef] [PubMed]

## 2. Image formation and reconstruction in SIM

*n*

^{th}SIM raw image can be generally written as where

*r⃗*denotes both sample as well as image coordinates (assuming a magnification of one).

*S*(

*r⃗*) denotes the sample fluorophore density, ⊗ the convolution operator,

*⊗*the imaginary unit,

*p⃗*the base pattern frequency and

*M*the number of harmonics in the periodic intensity pattern.

*a*is the strength of the

_{m}*m*harmonic and determines the pattern contrast.

^{th}*ϕ*is the sought-after pattern phase of the

_{n}*n*

^{th}image. In Fourier space Eq. (1) becomes where ∼ denotes the Fourier transform of a function and

*C̃*(

_{m}*k⃗*) are the different information components.

*D̃*(

_{n}*k⃗*) as elements of an image vector

*C̃*(

_{m}*k⃗*) as elements of a vector

**M**are defined as If the pattern phases are known, the components can be retrieved from the raw images by a simple inversion of Eq. (3): For isotropic resolution enhancement the process of image acquisition and component separation has to be repeated for a total of

*Q*orientations of the pattern (indicated by pattern vectors

*p⃗*) leading to separated components

_{q}*C̃*

_{q}_{,}

*(*

_{m}*k⃗*), where the additional index

*q*indicates the orientation used.

5. K. Wicker, O. Mandula, G. Best, R. Fiolka, and R. Heintzmann, “Phase optimisation for structured illumination microscopy,” Opt. Express **21**, 2032–2049 (2013). [CrossRef] [PubMed]

6. M. G. L. Gustafsson, L. Shao, P. M. Carlton, C. J. R. Wang, I. N. Golubovskaya, W. Z. Cande, D. A. Agard, and J. W. Sedat, “Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination,” Biophys. J. **94**, 4957–4970 (2008). [CrossRef] [PubMed]

*w*is the wiener filter parameter and

*Ã*(

*k⃗*) is the apodisation filter.

## 3. Current phase optimisation strategies

*p⃗*in Fourier space [4

4. S. A. Shroff, J. R. Fienup, and D. R. Williams, “Phase-shift estimation in sinusoidally illuminated images for lateral superresolution,” JOSA A **26**, 413–424 (2009). [CrossRef] [PubMed]

**M**

^{−1}used for component separation [5

**21**, 2032–2049 (2013). [CrossRef] [PubMed]

*et al*. Furthermore, as with most minimisation problems, it can potentially yield wrong phases as a result of getting stuck in a local minimum.

## 4. Single-step correlative phase determination

*D̃′*(

_{n}*k⃗*) =

*D̃*(

_{n}*k⃗*)

*h̃*

^{*}(

*k⃗*). We then analyse the auto-correlation (denoted by ⍟) of this filtered Fourier image at frequency

*p⃗*. Using Eq. (2) we get

**21**, 2032–2049 (2013). [CrossRef] [PubMed]

*m′*=

*m*+ 1. In this case we are correlating two shifted sample components which have a relative frequency distance of

*p⃗*and we are analysing this correlation also at frequency

*p⃗*, where there is thus a strong correlation. The integral term becomes which is purely real and positive. For the second case,

*m′*≠

*m*+ 1, this is different. We get Here we are correlating shifted sample components which have a relative distance not equal to

*p⃗*, but we are still looking at the correlation at

*p⃗*, which should thus be low. Although its expectation value is not zero, but rather depends on the auto-correlation function of the Fourier transformed sample, it is low enough compared to the strong correlations for

*m′*=

*m*+ 1 so that we can ignore its contribution to Eq. (7). Considering hence only the

*m′*=

*m*+ 1 terms, this equation can be simplified to As the sum is real and positive we can retrieve the pattern phase in the

*n*

^{th}image from the argument (or angle) of the complex valued

*𝒟*:

_{n}### 4.1. Periodic samples

*𝒞*

_{m}_{,}

_{m′}_{≠}

_{m}_{+1}is negligible compared to

*𝒞*

_{m}_{,}

_{m}_{+1}may not always be fulfilled. This can be the case for samples which have strong periodic structures with frequencies corresponding to those present in the illumination pattern, i.e.,

*mp⃗*. Here most algorithms will have difficulties differentiating between illumination and sample structures and yield results biased towards the phase of the sample structure. This situation should ideally be avoided, e.g., by slightly rotating the illumination pattern or the sample. If this is not feasible, the impact of this periodic sample structure may be partially alleviated by estimating its contribution through an auto-correlation of the OTF-filtered wide-field image at frequency

*p⃗*. This wide-field image can in turn be approximated as an average of all acquired raw images. This way the

*𝒞*

_{0,0}contribution can be avoided. However, for most samples this will not be necessary.

### 4.2. Determining changes in pattern contrast

*M*= 1) Eq. (10) becomes because

*a*

_{−1}

*a*

_{0}=

*a*

_{0}

*a*

_{1}and

*𝒞*

_{−1,0}=

*𝒞*

_{0,1}. Assuming a constant total image intensity

*a*

_{0}, any change in the pattern contrast

*c*= 2

*a*

_{1}can be detected by comparing the change in magnitude of

*𝒟*between different raw images: Pattern contrast does not usually change for different pattern phases. There are however a few cases where the contrast can change. Examples are non-linear SIM using fluorescence saturation [7

_{n}7. R. Heintzmann, T. M. Jovin, and C. Cremer, “Saturated patterned excitation microscopy - a concept for optical resolution improvement,” JOSA A **19**, 1599–1609 (2002). [CrossRef]

8. M. G. L. Gustafsson, “Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution,” PNAS **102**, 13081–13086 (2005). [CrossRef] [PubMed]

9. K. Wicker and R. Heintzmann, “Single-shot optical sectioning using polarization-coded structured illumination,” J. Opt. **12**, 084010 (2010). [CrossRef]

## 5. Performance of the algorithm

**21**, 2032–2049 (2013). [CrossRef] [PubMed]

### 5.1. Simulation parameters

*λ*= 488 nm; emission wavelength

_{ex}*λ*= 515 nm; numerical aperture

_{em}*NA*= 1.4; refractive index of the embedding medium

*n*= 1.52. This corresponds to a maximum detectable spatial frequency of (184 nm)

_{r}^{−1}, defined by the detection OTF. The pixel size in the simulated raw image corresponds to 65 nm in sample space. The synthetic sample used for the simulations is shown in Fig. 1. No camera offset or other background was assumed in the simulation.

*M*= 1) illumination. As in [5

**21**, 2032–2049 (2013). [CrossRef] [PubMed]

*p⃗*|: a fine pattern with a spatial frequency of |

*p⃗*| = 2

*π*(210 nm)

^{−1}, or 87.6% of the maximum frequency supported by the OTF; and a very fine pattern with a spatial frequency of |

*p⃗*| = 2

*π*(185 nm)

^{−1}, or 99.4% of the OTF support.

*ϕ*

_{1,}

*, 120° +*

_{p}*ϕ*

_{2,}

*, 240° +*

_{p}*ϕ*

_{3,}

*},*

_{p}*p*= 1..20. For each of these, I simulated noisy images for 51 different signal-to-noise levels, using Poisson noise and an expectation value of 10

^{l}^{/10},

*l*= 0..50, i.e., between 1 and 10

^{5}photons, in the brightest pixel. The raw images generated this way were the same as the ones used in [5

**21**, 2032–2049 (2013). [CrossRef] [PubMed]

### 5.2. Analysis

**D̃**

_{q}_{,}

_{p}_{,}

*= {*

_{l}*D̃*

_{1,}

_{q}_{,}

_{p}_{,}

*,*

_{l}*D̃*

_{2,}

_{q}_{,}

_{p}_{,}

*,*

_{l}*D̃*

_{3,}

_{q}_{,}

_{p}_{,}

*} (i.e., three raw SIM images for one pattern orientation (index*

_{l}*q*), with one set of randomised phases (index

*p*), for one particular photon level (index

*l*)) rather than for individual images. This was done according to [5

**21**, 2032–2049 (2013). [CrossRef] [PubMed]

**D̃**

_{q}_{,}

_{p}_{,}

*I subtracted the dataset’s known real phases from the phases determined by the weighted autocorrelations, yielding the individual remaining phase errors Δ*

_{l}*ϕ*

_{1,}

_{q}_{,}

_{p}_{,}

*, Δ*

_{l}*ϕ*

_{2,}

_{q}_{,}

_{p}_{,}

*and Δ*

_{l}*ϕ*

_{3,}

_{q}_{,}

_{p}_{,}

*. I then calculated the standard deviation of these three individual remaining phase errors,*

_{l}*denotes the mean of an expression over the index*

_{n}*n*. This

*ε*

_{q}_{,}

_{p}_{,}

*is called the phase error of the dataset*

_{l}**D̃**

_{q}_{,}

_{p}_{,}

*. This approach disregards any global pattern phase offsets, which is only relevant for the iterative approach, as it determines relative phase steps between image rather than the absolute phase of every individual image. This phase error is the measure of how well the phases were found for the particular dataset*

_{l}**D̃**

_{q}_{,}

_{p}_{,}

*. For each of the 51 photon levels (*

_{l}*l*) I then calculated the mean of the datasets’ phase errors over all 20 different phase variations (

*p*) and three orientations (

*q*),

*E*= 〈

_{l}*ε*

_{q}_{,}

_{p}_{,}

*〉*

_{l}

_{q}_{,}

*, calling this the*

_{p}*average phase error*for a particular photon level (

*l*). As a measure of robustness, I furthermore calculated the standard deviation from this average phase error,

### 5.3. Results

*E*of the new non-iterative (

_{l}*single step*) approach, with the shaded red area marking its standard deviation Δ

*E*. For comparison the dark blue line and shaded blue area show the results for the

_{l}*iterative*approach, corresponding to the data published in [5

**21**, 2032–2049 (2013). [CrossRef] [PubMed]

*λ*/800. Furthermore, its significantly lower standard deviation indicates a much stronger robustness of the single step approach, which unlike the iterative approach never produced any outliers.

*λ*/1000. Here, too, the much smaller standard deviation indicates a greater robustness of the single step approach, which again did not produce any outliers.

## 6. Reconstruction of experimental data

10. G. Best, R. Amberger, D. Baddeley, T. Ach, S. Dithmar, R. Heintzmann, and C. Cremer, “Structured illumination microscopy of autofluorescent aggregations in human tissue,” Micron **42**, 330–335 (2011). [CrossRef]

*λ*= 488 nm; emission wavelength

_{ex}*λ*> 500 nm; numerical aperture

_{em}*NA*= 1.4, oil immersion. SIM patterns were generated using two-beam illumination with frequencies |

*p⃗*| = {(310.6 nm)

^{−1}, (311.4 nm)

^{−1}, (301.5 nm)

^{−1}} for the three orientations

*α*= {0.9°, 120.8°, 59.7°}. For comparison I reconstructed the acquired data using a wrong assumption of equidistant phase steps and no optimisation, optimisation using the iterative approach and optimisation using the single step approach.

## 7. Discussion

*a posteriori*from the data. Through correlative approaches the pattern phase can be determined even in situations where the pattern is no longer visibly detectable in the acquired raw images (e.g., for extremely fine gratings or low signal-to-noise levels). I have presented a non-iterative approach which determines the pattern phase through weighted auto-correlations of individual raw Fourier images. Analysis of simulated SIM datasets shows that this approach yields accurate results with high reliability under realistic signal-to-noise conditions. Images reconstructed using this technique could not be visibly discerned from images reconstructed using the iterative approach [5

**21**, 2032–2049 (2013). [CrossRef] [PubMed]

*N*autocorrelations, whereas the optimised iterative approach requires the calculation of

*N*

^{2}

*L*auto- and cross-correlations before the start of the iterative optimisation,

*N*being the number of raw images and

*L*the largest whole number for which the shifted OTF

*h̃*(

*k⃗*−

*Lp⃗*) still has overlapping support with the unshifted OTF

*h̃*(

*k⃗*).

## Acknowledgments

## References and links

1. | R. Heintzmann and C. Cremer, “Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating,” Proc. SPIE |

2. | M. G. L. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. |

3. | J. T. Frohn, “Super-resolution fluorescence microscopy by structured light illumination,” Ph.D. thesis, Eidgenössische Technische Hochschule Zürich, Switzerland (2000). |

4. | S. A. Shroff, J. R. Fienup, and D. R. Williams, “Phase-shift estimation in sinusoidally illuminated images for lateral superresolution,” JOSA A |

5. | K. Wicker, O. Mandula, G. Best, R. Fiolka, and R. Heintzmann, “Phase optimisation for structured illumination microscopy,” Opt. Express |

6. | M. G. L. Gustafsson, L. Shao, P. M. Carlton, C. J. R. Wang, I. N. Golubovskaya, W. Z. Cande, D. A. Agard, and J. W. Sedat, “Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination,” Biophys. J. |

7. | R. Heintzmann, T. M. Jovin, and C. Cremer, “Saturated patterned excitation microscopy - a concept for optical resolution improvement,” JOSA A |

8. | M. G. L. Gustafsson, “Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution,” PNAS |

9. | K. Wicker and R. Heintzmann, “Single-shot optical sectioning using polarization-coded structured illumination,” J. Opt. |

10. | G. Best, R. Amberger, D. Baddeley, T. Ach, S. Dithmar, R. Heintzmann, and C. Cremer, “Structured illumination microscopy of autofluorescent aggregations in human tissue,” Micron |

**OCIS Codes**

(100.2000) Image processing : Digital image processing

(100.6640) Image processing : Superresolution

(180.2520) Microscopy : Fluorescence microscopy

**ToC Category:**

Image Processing

**History**

Original Manuscript: August 16, 2013

Revised Manuscript: September 17, 2013

Manuscript Accepted: September 25, 2013

Published: October 8, 2013

**Citation**

Kai Wicker, "Non-iterative determination of pattern phase in structured illumination microscopy using auto-correlations in Fourier space," Opt. Express **21**, 24692-24701 (2013)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-21-21-24692

Sort: Year | Journal | Reset

### References

- R. Heintzmann and C. Cremer, “Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating,” Proc. SPIE3568, 185–196 (1999). [CrossRef]
- M. G. L. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc.198, 82–87 (2000). [CrossRef] [PubMed]
- J. T. Frohn, “Super-resolution fluorescence microscopy by structured light illumination,” Ph.D. thesis, Eidgenössische Technische Hochschule Zürich, Switzerland (2000).
- S. A. Shroff, J. R. Fienup, and D. R. Williams, “Phase-shift estimation in sinusoidally illuminated images for lateral superresolution,” JOSA A26, 413–424 (2009). [CrossRef] [PubMed]
- K. Wicker, O. Mandula, G. Best, R. Fiolka, and R. Heintzmann, “Phase optimisation for structured illumination microscopy,” Opt. Express21, 2032–2049 (2013). [CrossRef] [PubMed]
- M. G. L. Gustafsson, L. Shao, P. M. Carlton, C. J. R. Wang, I. N. Golubovskaya, W. Z. Cande, D. A. Agard, and J. W. Sedat, “Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination,” Biophys. J.94, 4957–4970 (2008). [CrossRef] [PubMed]
- R. Heintzmann, T. M. Jovin, and C. Cremer, “Saturated patterned excitation microscopy - a concept for optical resolution improvement,” JOSA A19, 1599–1609 (2002). [CrossRef]
- M. G. L. Gustafsson, “Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution,” PNAS102, 13081–13086 (2005). [CrossRef] [PubMed]
- K. Wicker and R. Heintzmann, “Single-shot optical sectioning using polarization-coded structured illumination,” J. Opt.12, 084010 (2010). [CrossRef]
- G. Best, R. Amberger, D. Baddeley, T. Ach, S. Dithmar, R. Heintzmann, and C. Cremer, “Structured illumination microscopy of autofluorescent aggregations in human tissue,” Micron42, 330–335 (2011). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.