2. HiLo microscopy basics
HiLo microscopy requires two images to obtain one optically sectioned image. A uniform-illumination image (iu
) is used to obtain the high-frequency (HI) components by means of a digital high-pass filter and a second image (is
), obtained with speckle-illumination, is used to identify the axially resolved low-frequency (LO) components of the object. The combination of these two images provides a full-frequency axially sectioned image.
Optical sectioning in HiLo microscopy is obtained by processing the HI and LO regions of the spatial spectrum of the object in different ways. The procedure used to retrieve the HI in-focus components is based on typical properties of the optical transfer function (OTF) of a standard wide-field microscope. HI components of an object are well resolved only when the object is in focus, while LO components remain visible even if it is out of focus [
6J. W. Goodman, Introduction to Fourier Optics (2nd Ed., McGraw-Hill, 1996), Chap. 6.
]. Therefore, HI components are naturally optically sectioned and they can be extracted from
iu
simply using a high-pass Fourier filter (
HP). The in-focus HI components (
ihi
) are obtained as
, where
stands for the inverse Fourier transform operation,
Iu
is the Fourier transform of
iu
,
ρ are spatial coordinates of the image and
HP is a Gaussian high-pass filter with cutoff frequency
κc
, such that
for all
.
In order to retrieve the in-focus LO components, a speckle pattern is used to illuminate the sample [
1
D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy,” Opt. Lett.
33(16), 1819–1821 (2008). [CrossRef] [PubMed]
,
7
C. Ventalon and J. Mertz, “Quasi-confocal fluorescence sectioning with dynamic speckle illumination,” Opt. Lett.
30(24), 3350–3352 (2005). [CrossRef] [PubMed]
]. The OTF of the microscope determines that small details of the illumination (high frequency components) yield high image contrast only if they are originated in the in-focus plane of the object; namely an optical section. The speckled epi-fluorescence from all out-of-focus sections present low image contrast. Hence, only the regions of the image that show high speckle contrast correspond to the in-focus axial plane, the rest is out-of-focus background. Therefore, the contrast in
is
is an indicator of in-focus information, which is calculated locally within square sampling windows of side Λ throughout the image, according to
where
and
are the standard deviation and the mean value of
is
respectively, computed locally within each sampling window. Since only the intensity variations caused by speckle are sought,
CS
should be corrected to eliminate the variations due to the object itself [
1
D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy,” Opt. Lett.
33(16), 1819–1821 (2008). [CrossRef] [PubMed]
]. An alternative approach consists on calculating the local contrast on the image difference
is-iu
, as it was recently proposed [
3
D. Lim, T. N. Ford, K. K. Chu, and J. Mertz, “Optically sectioned in vivo imaging with speckle illumination HiLo microscopy,” J. Biomed. Opt.
16(1), 016014 (2011). [CrossRef] [PubMed]
]. Finally,
CS
is applied as a weighting function on
iu
to obtain
which is a low-resolution image of the in-focus plane of the object. In order to obtain the LO components of the object that belong only to a frequency range that is complementary to
ihi
, a complementary low-pass filter
LP = 1 - HP is applied as
, where
Isu
is the Fourier transform of
isu
. The final image, containing the in-focus information of the full spatial frequency range, is computed as
, where the parameter
η balances both intensities for avoiding discontinuities at
κc
in the spatial spectrum of
iHiLo
[
1
D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy,” Opt. Lett.
33(16), 1819–1821 (2008). [CrossRef] [PubMed]
,
3
D. Lim, T. N. Ford, K. K. Chu, and J. Mertz, “Optically sectioned in vivo imaging with speckle illumination HiLo microscopy,” J. Biomed. Opt.
16(1), 016014 (2011). [CrossRef] [PubMed]
].
It should be noted that the size of the window used to calculate Cs
determines the sampling of isu
. Therefore, the maximum spatial frequency component that is present in isu
is 1/2Λ and in order to obtain the full spatial spectrum of the object, the cutoff frequency of LP and HP should be such that
Finally, the axial resolution of HiLo can be significantly increased by applying a band-pass filter as
to the speckle-illuminated image before computing
. In fact, the axial resolution can be tuned by changing the parameter
σw
and setting
κc
to approximately 0.18
σw
[
3
D. Lim, T. N. Ford, K. K. Chu, and J. Mertz, “Optically sectioned in vivo imaging with speckle illumination HiLo microscopy,” J. Biomed. Opt.
16(1), 016014 (2011). [CrossRef] [PubMed]
].
3. Speckle contrast and image quality
From
Eq. (2) it follows that the local speckle contrast
CS
determines the characteristics of the final image
iHiLo
and the whole methodology relies heavily on accurately measuring and processing such speckle contrast. Under certain conditions, undesired features of the speckle contrast induce artifacts in
iHiLo
so that local brightness and particle density measurements can become biased.
The local contrast of speckle patterns presents a typical variation that is inherent to its random nature. For example, if a thin uniform object is imaged using fully developed speckle illumination, the contrast at the sample is approximately 1. However, due to the inevitable filtering caused by the imaging optics, the observed contrast at the camera plane takes an overall value that is significantly lower than 1. The local contrast
CS
computed in small regions randomly changes from one region to another and, in fact, it is distributed according to log-normal statistics [
8
D. D. Duncan, S. J. Kirkpatrick, and R. K. Wang, “Statistics of local speckle contrast,” J. Opt. Soc. Am. A
25(1), 9–15 (2008). [CrossRef] [PubMed]
]. As a result, even for uniform samples, the contrast is not uniform throughout the visual field and an artifact is propagated to the final image; regions of high contrast appear brighter than regions of low contrast. As stated by Duncan et al. [
8
D. D. Duncan, S. J. Kirkpatrick, and R. K. Wang, “Statistics of local speckle contrast,” J. Opt. Soc. Am. A
25(1), 9–15 (2008). [CrossRef] [PubMed]
], the dispersion of the local contrast depends on the side
Λ of the sampling window and on the characteristic width of the speckle grains (
Δg). However, as it will be shown below, the parameter that determines the dispersion of the local contrast is the ratio
Λ/Δg, which represents the square root of the number of speckle grains that fit in each sampling window.
Considering that speckle grains are diffraction limited at the sample, their size in the image plane is approximately determined by the convolution of the illumination and detection squared point spread functions. As an approximation we define
Δg~1.22 λ /
NA, where λ is the emission wavelength and
NA is the numerical aperture of the microscope objective. The fact that
κmax = 2 NA / λ is the maximum spatial frequency allowed by the microscope optics, and that
κc
was chosen equal to
1/2Λ (see
Eq. (3)), the ratio
Λ/Δg is related with the cutoff frequency
κc
, according to
As the dispersion of contrast values is determined by the ratio
Λ/Δg,
Eq. (5) indicates that such artifact can be smoothed by properly choosing the value of
κc
. A thorough selection of these parameters is key to assure a good image quality and a satisfactory outcome.
We analyzed this effect using both experimental data and numerical simulations. We defined a “roughness” parameter
R calculated as the standard deviation of
CS
within the whole image, expressed as a percentage of its mean value
<CS>, i.e.
and we numerically analyzed how
R depends on the sampling window size (
Λ). To do that, a planar uniform object
, illuminated with fully developed speckle of characteristic size
Δg = 1.5μm (
λ = 0.5μm,
NA = 0.4) was synthesized. Speckle illumination was simulated using the Fresnel diffraction theory of coherent light using real parameters of the system and image formation in the CCD was computed using incoherent propagation theory [
5
S. Santos, K. K. Chu, D. Lim, N. Bozinovic, T. N. Ford, C. Hourtoule, A. C. Bartoo, S. K. Singh, and J. Mertz, “Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle,” J. Biomed. Opt.
14(3), 030502 (2009). [CrossRef] [PubMed]
,
9J. Goodman, Speckle phenomena in optics (Roberts & Company, 2007).
].
In
Fig. 1(b)
, we show the roughness
R of the image calculated as a function of
Λ. The ratio
Λ/Δg is displayed in the top scale while the bottom scale shows |
κc
|/|
κmax
|, as determined by
Eq. (5). For each value of
Λ/Δg, 10
4 non-overlapping regions were computed and the same simulation was repeated using speckle grains corresponding to different
NA objectives (0.75 and 0.95), showing that the ratio
Λ/Δg is the right parameter to describe
R.
Fig. 1 (a) Scheme of HiLo microscope. A laser diode at 473nm (Laserglow, Toronto, CA) illuminates a ground glass diffuser. Two lenses L1 and L2 of 100mm and 300mm focal lengths are used to illuminate the back-focal-plane of an objective in an inverted fluorescence microscope. A CCD camera (Retiga 2000R) was used for imaging. (b) Output roughness as a function of |κc
|/|κmax
| (bottom axis). The parameter Λ/Δg, in the top axis, roughly represents the square root of the number of speckle grains that fits in each sampling window.
This result indicates that to minimize this artifact, in which intensity variations that are not originated in the sample arise, the number of speckle grains within the sampling window should be increased. Nevertheless, if Λ is increased, |κc
| is reduced and consequently the axial resolution decreases. More specifically, since the axial resolution of HiLo heavily relies on processing the LO components using structured illumination, as |κc
| is reduced HiLo approaches the behavior of a wide-field microscope. Alternatively, if speckle grains are shrunk by increasing the NA of the objective, lateral resolution is also improved, but field of view (FOV) is reduced. Given that lateral resolution in HiLo microscopy is independent of |κc
| (it is only limited by diffraction), for admissible values of image roughness, the trade-off to consider is between axial resolution and FOV.
In order to reproduce these results experimentally, we have built a HiLo setup, where the speckle illumination was implemented through the lamp port of an inverted microscope using a ground glass diffuser, a blue diode laser and a lens [
1
D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy,” Opt. Lett.
33(16), 1819–1821 (2008). [CrossRef] [PubMed]
,
7
C. Ventalon and J. Mertz, “Quasi-confocal fluorescence sectioning with dynamic speckle illumination,” Opt. Lett.
30(24), 3350–3352 (2005). [CrossRef] [PubMed]
], as depicted in
Fig. 1(a). The position of the diffuser was conjugated to the back-focal-plane of the objective and uniform illumination images were obtained by rotating the diffuser with a DC motor.
We fabricated a sample consisting of several squared monolayers of fluorescent proteins of 20×20 μm
2 using a method that allows printing protein patterns of arbitrary shapes and concentrations with micron resolution [
10
J. M. Bélisle, J. P. Correia, P. W. Wiseman, T. E. Kennedy, and S. Costantino, “Patterning protein concentration using laser-assisted adsorption by photobleaching, LAPAP,” Lab Chip
8(12), 2164–2167 (2008). [CrossRef] [PubMed]
,
11
J. M. Belisle, D. Kunik, and S. Costantino, “Rapid multicomponent optical protein patterning,” Lab Chip
9(24), 3580–3585 (2009). [CrossRef] [PubMed]
]. For our experiments we produced squared uniform distributions of Avidin-Fluorescein and the sample was imaged using a 60X 1.35
NA objective.
HiLo images of the same fluorescent pattern obtained with different values of
Λ and a standard widefield image are displayed in the four panels of
Fig. 2(a)
. As can be observed, the roughness artifact is efficiently smoothed by increasing
Λ without distorting lateral resolution. Furthermore, in
Fig. 2(b) intensity profiles of the images displayed in
Fig. 2(a) are traced, where the reduction of the artifact is also evident. For quantifying this effect, the experimental roughness within the central region (15×15μm
2) of the image, was computed on
iHiLo
similarly to
Eq. (6). The values of
R(
iHiLo
) are plotted in
Fig. 2(c) for different sizes of the window, showing that it strongly depends on
Λ/Δg as calculated with numerical simulations in
Fig. 1(b).
Fig. 2 20×20μm2 planar object imaged in-focus with a 60X NA=1.35 objective. (a) HiLo Images obtained using sampling windows with Λ/Δg of 1.3, 4 and 8 as indicated. In the experiment Δg = 0.46μm. The image obtained with uniform illumination is also shown. (b) Intensity profiles of images in (a) traced over the vertical dashed line P. (c) Roughness of the experimental image for various values of Λ/Δg (upper scale). The corresponding values of |κc
|/|κmax
| are shown in the bottom scale. The dashed red line represents the roughness of iu
.
This effect must be taken into consideration when planning biological imaging. It is not problematic for 3-dimensional reconstructions, image segmentation, fluorescence recovery after photobleaching, measuring fluorescent protein expression levels, etc. Nevertheless, for single particle tracking, image correlation spectroscopy or colocalization studies, it must be taken into account.
4. Speckle contrast and optical sectioning
A second issue we considered is the situation of thick objects. In these cases, the out-of-focus fluorescence can be high enough to change the speckle contrast yielding non-intuitive results. On one hand, when the fluorescence background is very intense, the contrast is deeply reduced. In conditions in which speckle contrast is so low that cannot be distinguished from noise, Cs becomes a flat function that doesn’t discriminate signal from background. On the other hand, strong fluctuations in the object thickness, produce fluctuations in Cs that propagate to the final HiLo image showing an artificial variation of the intensity, which does not reflect a change of fluorophore concentration inherent to the optical section imaged. As opposed to confocal laser scanning and multiphoton microscopies, which measure fluorescence signals that originate only at in-focus plane, HiLo uses information that arises from out-of-focus regions too, and this fact must be considered in the analysis. As the out-of-focus fluorescence affect the image contrast, this effect is not specific to speckle illumination and could be generalized to other structured illumination approaches that rely on the analysis of the contrast to obtain optical sectioning. However, the specific impact on each particular technique should be separately studied.
The out of focus sections of the sample produce a fluorescence background that ultimately affects HiLo images in a non-intuitive way. Speckle illumination consists of a pattern of grains in which size remains relatively constant along the propagation direction of the beam, as depicted in
Fig. 1(a). Therefore, when a fluorescent object thicker than the focal volume is illuminated like this, the epi-fluorescence detected combines the signal originating from speckle grains located at different depths. While speckle grains near the object plane yield high contrast, the grains located off-plane produce a defocused background that lessens the values of Cs.
Summarizing, an increase in out-of-focus fluorescence background produces an overall decrease of
Cs, since off-plane fluorescence contributes low speckle fluctuations but increases the total signal. In fact,
Cs acts as a weighting function that highlights the signal originated from the in-focus regions over those out-of-focus, but the strength of such enhancement becomes limited when
CS
is too small. Even in the absence of speckle, acquisition noise (shot noise, readout noise, etc) produces an intensity contrast (
CN
). Albeit the mean value of
CN
can be numerically reduced to zero [
1
D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy,” Opt. Lett.
33(16), 1819–1821 (2008). [CrossRef] [PubMed]
,
3
D. Lim, T. N. Ford, K. K. Chu, and J. Mertz, “Optically sectioned in vivo imaging with speckle illumination HiLo microscopy,” J. Biomed. Opt.
16(1), 016014 (2011). [CrossRef] [PubMed]
], the fluctuations around the mean cannot be avoided and when, in strong background conditions, the speckle fluctuations are minimized, the capacity to distinguish in-focus from out-of-focus information is lost. When speckle fluctuations originated from in-focus regions become similar to noise, optical sections cannot be obtained, and this effect ultimately imposes a limit to the characteristics of objects that can be analyzed using HiLo microscopy.
In order to study the dependence of contrast on the axial thickness of the sample in more detail, we combined experimental data and numerical simulations, as described above. We first computed numerically the illumination pattern produced by a diffuser to show the 3D structure of speckle grains and three examples at different depths are shown in
Fig. 3(a)
. To simulate the speckle illumination image produced by a thick fluorescent object in the CDD, we calculated the incoherent propagation of all object planes at different depths to the detector. The final image was obtained by summing up all such propagated intensities. This procedure was repeated for samples of different thicknesses and in each case the overall contrast was computed in the central region of the final image. The curves in
Fig. 3(b) represent the simulated contrast versus the axial thickness
T of the sample, where each line corresponds to a different
NA objective.
Fig. 3 (a) Numerical simulation of speckle produced with λ = 488 nm and NA=0.4 in 3 axial planes. (b) Contrast of speckle illuminated sample (c) at various axial widths. Each position of the sample was illuminated with objectives of 3 different NA. The discrete markers correspond to the experimental measurements while the lines represent numerical simulations. Estimated experimental uncertainties are smaller than the markers size. Note that experiment and numerical simulation results are plotted with different scales. (c) Scheme of wedge-shaped glass chamber filled with fluorescent solution. The sample was imaged in regions I and II. (d) Scheme depicting the position of the object plane in a region of sample in (c).
Experimentally, we tested this effect by fabricating a fluorescent sample of variable thickness as depicted in
Fig. 3(c). It consists of a wedge-shaped glass chamber made with coverslips and filled with a fluorescent dye solution (fluorescein in water and 10% methanol). Objects of different thicknesses were obtained by simply moving transversally the sample with respect to the microscope objective. At each position on the sample, the thickness was measured using a homemade Fourier-domain optical coherent tomography device coupled to the lateral port of the microscope [
12
K. Singh, C. Dion, S. Costantino, M. Wajszilber, M. R. Lesk, and T. Ozaki, “Development of a novel instrument to measure the pulsatile movement of ocular tissues,” Exp. Eye Res.
91(1), 63–68 (2010). [CrossRef] [PubMed]
,
13
K. Singh, C. Dion, M. R. Lesk, T. Ozaki, and S. Costantino, “Spectral-domain phase microscopy with improved sensitivity using two-dimensional detector arrays,” Rev. Sci. Instrum.
82(2), 023706 (2011). [CrossRef] [PubMed]
]. The contrast obtained for each position is plotted with discrete markers in
Fig. 3(b) and a good match between the experiments and numerical simulation can be observed (a 1.4 scaling factor was used).
As shown in
Fig. 3(b), for a given sample thickness, the contrast is higher for low
NA objectives. The reason for this is that when objects are illuminated with high
NA objectives, speckle grains are small, the Rayleigh range is narrow, and therefore the fraction of out-of-focus fluorescence measured is high. To mitigate this effect, the size of speckle grains must be increased, which can be achieved by reducing the
NA of the objective. In this case, a trade-off between the sample width and the lateral resolution should be considered. An alternative approach is to reduce only the NA of the illumination. This could be done by placing an iris in the illumination pathway but not in the detection pathway (i.e. before the dichroic beam-splitter). Thereby, the size of the grains is increased without compromising lateral resolution but reducing the FOV; there is a trade-off between sample width and FOV.
As stated above, HiLo images depend on
Cs, but more specifically on the product of
Cs and
iu
(see
Eq. (2)). If we consider the sample depicted in
Fig. 3, we expect an optical section to render a constant intensity image, since the dye concentration is homogeneous at any depth. The value of
Cs is reduced as the axial thickness of the sample increases, but the widefield intensity
iu
increases. However, it is not clear if these variations would compensate, and in general whether the HiLo image is uniform.
To experimentally illustrate this situation, we fabricated samples like the ones in
Fig. 3(c) having large angles to obtain substantial variations of the thickness within the FOV. Two regions of the sample (I and II) were measured as indicated in the figure. The total thickness variation within the FOV is approximately the same (40μm) in both regions, but the mean thickness in region I is 40μm and in region II is 160μm. The corresponding experimental results are presented in
Fig. 4
. The top plots on the right column show
iu
vs.
T(
x) along with their linear fits. In both cases, the intercepts of both linear fits are approximately 0, consistent with the fact that the intensity vanishes as T→0. For each region, the images corresponding to
is
,
iu
,
Cs and
iHiLo
are presented in the left, and the averaged horizontal profiles are plotted with black markers on the right.
Fig. 4 Images of a wedge-shaped chamber filled with dye solution in two regions: I is a thin region of main thickness ~40μm and II is a thicker region of main thickness ~160μm. Each panel is organized as follows: In the left column the images is
, iu
, Cs and iHiLo
are shown while the three bottom plots in the right column show the corresponding average horizontal profiles (notice the transversal coordinate is displayed in the top scale and the bottom scale indicates thickness of the sample). The profiles of iHiLo
including band-pass prefiltering with σw=κmax
, κmax
/2 and κmax
/3 are displayed (with independent normalization factors) along with iHiLo
obtained without prefiltering. The top graph of each right column shows the linear fit of iu
vs. the thickness T demonstrating the linearity of the CCD in the measured range. iHiLo
without band-pass filter was computed with Λ=11 pixels, and |κc|=1/2Λ. The objective is a 10X 0.4NA, and the CCD pixel size is 7.4 μm.
The results show that the changes on
Cs and
iu
, due to variations in the out-of-focus background, do propagate to the final
iHiLo
image yielding optical sections of non-constant intensities. These variations are more dramatic in region I, where
Cs decreases so steeply with
T that the linear increase of
iu
is clearly not enough to compensate for it. Besides, in the graph of
iHiLo
for region I, a local maximum can be observed near
x=100μm. This point (A) corresponds to the position where the object in-focus plane intercepts the edge of the sample, as schematized in
Fig. 3(d), and for
x<A,
Cs(
x) decreases as the object gets out of focus, while for
x>A a part of the object is in focus, but
Cs(x) decreases as the object thickness increases. Indeed, the
Cs peak shifts accordingly in
x, as the object plane is axially displaced (data not shown), confirming this explanation.
In region II, the variation in Cs(x) is smaller than in Region I, so that it is approximately compensated by the increase in iu
yielding a rather uniform iHiLo
optical section. Interestingly, the total thickness change in regions I and II is the same (40μm), but the effects of the relative variation ΔT/T in Cs and iu
are not inversely proportional.
We finally computed
iHiLo
using the pass-band filter of
Eq. (4) to assess its impact on the effect described above. The intensity profiles obtained with filters built with
σw
=
κmax
,
κmax
/2 and
κmax
/3 are plotted in the bottom graphs of
Fig. 4 along with the result without prefiltering, showing that this operation does not change the overall behavior regarding the non-constant intensity profiles produced by background fluctuations.
Overall, in contrast to what happens in confocal microscopy, since in-focus and out-of-focus fluorescence are used to calculate the optical sections, this combination needs to be understood for a correct interpretation of the HiLo images. The profile of the optical sections obtained with both techniques can be different and this fact must be considered.
It is worth mentioning, however, that typical biological samples are seldom as ubiquitously fluorescent as a dye solution. In consequence, in most cases the out of focus background will not produce such a dramatic effect. Nevertheless, if precise quantifications of the intensity are sought, this effect should be acknowledged.