OSA's Digital Library

Optics Express

Optics Express

  • Editor: C. Martijn de Sterke
  • Vol. 19, Iss. 18 — Aug. 29, 2011
  • pp: 17439–17452
« Show journal navigation

Depth-resolved image mapping spectrometer (IMS) with structured illumination

Liang Gao, Noah Bedard, Nathan Hagen, Robert T. Kester, and Tomasz S. Tkaczyk  »View Author Affiliations


Optics Express, Vol. 19, Issue 18, pp. 17439-17452 (2011)
http://dx.doi.org/10.1364/OE.19.017439


View Full Text Article

Acrobat PDF (1666 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

We present a depth-resolved Image Mapping Spectrometer (IMS) which is capable of acquiring 4D (x, y, z, λ) datacubes. Optical sectioning is implemented by structured illumination. The device’s spectral imaging performance is demonstrated in a multispectral microsphere and mouse kidney tissue fluorescence imaging experiment. We also compare quantitatively the depth-resolved IMS with a hyperspectral confocal microscope (HCM) in a standard fluorescent bead imaging experiment. The comparison results show that despite the use of a light source with four orders of magnitude lower intensity in the IMS than that in the HCM, the image signal-to-noise ratio acquired by the IMS is 2.6 times higher than that achieved by the equivalent confocal approach.

© 2011 OSA

1. Introduction

Fluorescence microscopy is an indispensable tool in biological research and has been used extensively in widespread applications [1

1. J. W. Lichtman and J. A. Conchello, “Fluorescence microscopy,” Nat. Methods 2(12), 910–919 (2005). [CrossRef] [PubMed]

]. The continuing development of fluorescent probes bringing new spectral characteristics and new functional properties for cellular labeling has widened the possibilities of what biological features can be measured by a microscope. Taking full advantage of their properties, however, often requires imaging systems have a spectral resolving power to discriminate those fluorescent probes which have significant spectral overlaps.

Hyperspectral imaging is a hybrid imaging modality that combines the advantages of traditional imaging cameras and spectrometers [2

2. Y. Hiraoka, T. Shimi, and T. Haraguchi, “Multispectral imaging fluorescence microscopy for living cells,” Cell Struct. Funct. 27(5), 367–374 (2002). [CrossRef] [PubMed]

]. It measures sample’s spatial and spectral information, and constructs a 3D (x, y, λ) datacube for spectral unmixing analysis [3

3. T. Zimmermann, J. Rietdorf, and R. Pepperkok, “Spectral imaging and its applications in live cell microscopy,” FEBS Lett. 546(1), 87–92 (2003). [CrossRef] [PubMed]

]. However, most hyperspectral imagers are scanning-based systems. They either scan in the spatial domain, e.g. hyperspectral confocal microscope [4

4. V. Raicu, M. R. Stoneman, R. Fung, M. Melnichuk, D. B. Jansma, L. F. Pisterzi, S. Rath, M. Fox, J. W. Wells, and D. K. Saldin, “Determination of supramolecular structure and spatial distribution of protein complexes in living cells,” Nat. Photonics 3(2), 107–113 (2009). [CrossRef]

] and slit-scanning microscope [5

5. S. Kumazaki, M. Hasegawa, M. Ghoneim, Y. Shimizu, K. Okamoto, M. Nishiyama, H. Oh-Oka, and M. Terazima, “A line-scanning semi-confocal multi-photon fluorescence microscope with a simultaneous broadband spectral acquisition and its application to the study of the thylakoid membrane of a cyanobacterium Anabaena PCC7120,” J. Microsc. 228(Pt 2), 240–254 (2007). [CrossRef] [PubMed]

], or scan in the spectral domain, e.g., liquid crystal tunable filters or acoustic optic tunable filters [6

6. H. R. Morris, C. C. Hoyt, and P. J. Treado, “Imaging spectrometers for fluorescence and Raman microscopy - acoustooptic and liquid-crystal tunable filters,” Appl. Spectrosc. 48(7), 857–866 (1994). [CrossRef]

]. Because scanning-based systems cannot collect light from all voxels of the datacube in parallel, there is a loss of light throughput by a factor of N when measuring N scan elements. To overcome this limitation, snapshot hyperspectral imagers such as Computed Tomography Imaging Spectrometer (CTIS) [7

7. B. K. Ford, C. E. Volin, S. M. Murphy, R. M. Lynch, and M. R. Descour, “Computed tomography-based spectral imaging for fluorescence microscopy,” Biophys. J. 80(2), 986–993 (2001). [CrossRef] [PubMed]

], Coded Aperture Snapshot Spectral Imager (CASSI) [8

8. C. F. Cull, K. Choi, D. J. Brady, and T. Oliver, “Identification of fluorescent beads using a coded aperture snapshot spectral imager,” Appl. Opt. 49(10), B59–B70 (2010). [CrossRef] [PubMed]

] and Image-Replicating Imaging Spectrometer (IRIS) [9

9. A. Gorman, D. W. Fletcher-Holmes, and A. R. Harvey, “Generalization of the Lyot filter and its application to snapshot spectral imaging,” Opt. Express 18(6), 5602–5608 (2010). [CrossRef] [PubMed]

] have been developed. Although the spectral imaging capabilities of these snapshot hyperspectral imagers have been demonstrated, they suffer many problems, e.g., CTIS and CASSI require extensive computational cost, while IRIS is limited in the number of spectral bands which can acquire at high spatial resolution.

The Image Mapping Spectrometer (IMS) is a novel snapshot hyperspectral imager that is developed for full-throughput measurement of spectrally resolved scenes [10

10. L. Gao, R. T. Kester, and T. S. Tkaczyk, “Compact image slicing spectrometer (ISS) for hyperspectral fluorescence microscopy,” Opt. Express 17(15), 12293–12308 (2009). [CrossRef] [PubMed]

12

12. R. T. Kester, N. Bedard, L. Gao, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J. Biomed. Opt. 16(5), 056005 (2011). [CrossRef] [PubMed]

]. It replaces the regular camera in a digital imaging system, allowing one to add spectrum acquisition capability to a variety of imaging modalities, such as microscopy [10

10. L. Gao, R. T. Kester, and T. S. Tkaczyk, “Compact image slicing spectrometer (ISS) for hyperspectral fluorescence microscopy,” Opt. Express 17(15), 12293–12308 (2009). [CrossRef] [PubMed]

,11

11. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy,” Opt. Express 18(14), 14330–14344 (2010). [CrossRef] [PubMed]

] and endoscopy [12

12. R. T. Kester, N. Bedard, L. Gao, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J. Biomed. Opt. 16(5), 056005 (2011). [CrossRef] [PubMed]

]. The operation of the IMS is based on the image mapping principle [10

10. L. Gao, R. T. Kester, and T. S. Tkaczyk, “Compact image slicing spectrometer (ISS) for hyperspectral fluorescence microscopy,” Opt. Express 17(15), 12293–12308 (2009). [CrossRef] [PubMed]

]. Briefly, a custom-fabricated optical component – termed image mapper – is utilized to remap a sample’s 3D (x, y, λ) datacube onto a 2D CCD camera, so that a sample’s spatial and spectral information can be measured in parallel. Since no scanning is employed, the IMS features high-speed (currently up to 7.2 fps [12

12. R. T. Kester, N. Bedard, L. Gao, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J. Biomed. Opt. 16(5), 056005 (2011). [CrossRef] [PubMed]

]) (x, y, λ) datacube acquisition without sacrificing spatial and spectral resolution. In addition, since the IMS is a direct imaging device, little computational cost is required to reconstruct a (x, y, λ) datacube.

Although the throughput of the IMS is halved by the added sectioning hardware (see Section 2), high contrast depth-resolved spectral channel images were still acquired in a mouse kidney tissue fluorescence imaging experiment (see Section 3). In addition, due to the added sectioning capability, a 4D (x, y, z, λ) datacube is also successfully acquired by the IMS. The 4D imaging results demonstrate that no artifacts are introduced by integrating traditional 3D (x, y, z) imaging with IMS’ multispectral imaging.

To show the advantages of depth-resolved IMS over other commercially available multispectral imagers, we quantitatively compare the depth-resolved IMS with a hyperspectral confocal microscope (HCM) (Zeiss Meta 510) in the context of photon collection and image signal-to-noise ratio (SNR) (see Section 4). The experimental results indicate that, at a given frame rate, the depth-resolved IMS surpasses the corresponding HCM approach 130 times in the photon collection, and 2.6 times in the image SNR respectively.

2. System layout

The schematic of the depth-resolved IMS is shown in Fig. 1
Fig. 1 Depth-resolved IMS with SI.
. The IMS is coupled to a side image port of an inverted microscope (Axio Observer, Zeiss). The optical design and layout for the IMS is detailed in [12

12. R. T. Kester, N. Bedard, L. Gao, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J. Biomed. Opt. 16(5), 056005 (2011). [CrossRef] [PubMed]

]. Here the IMS is assembled in a robust ruggedized enclosure so that the device is stable and functional in ambient room light. The 3D (x, y, λ) datacube that the IMS acquires is of size 350 × 350 × 46. The spatial sampling and spectral sampling are balanced in the IMS design so that over 90% IMS’ camera pixels can be effectively utilized [12

12. R. T. Kester, N. Bedard, L. Gao, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J. Biomed. Opt. 16(5), 056005 (2011). [CrossRef] [PubMed]

]. When a Zeiss Plan-Apochromat 63 × oil immersion objective (NA = 1.4) is used on the microscope, the IMS achieves ~0.44 μm lateral spatial resolution in the 160 μm × 160 μm field-of-view (FOV).

The sample is illuminated by a 120W X-cite arc lamp. Structured illumination is implemented by placing a 10 lp/mm Ronchi ruling grid (NT38-258, Edmund Optics) at a conjugate plane of the microscope’s sample plane in the epi-illumination optical path. The grid is mounted on a ceramic servo motor stage (ALIO industries, P/N: AI-HR4-10000E-XY). The IMS is synchronized with the movement of the grid for automatic acquisition. The depth-resolved (x, y, λ) datacube I is acquired from three (x, y, λ) datacubesI1, I2andI3which are captured with2π/3phase shifted sinusoid illumination patterns by the demodulation algorithm [13

13. M. A. A. Neil, R. Juskaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22(24), 1905–1907 (1997). [CrossRef] [PubMed]

,14

14. D. Karadaglić and T. Wilson, “Image formation in structured illumination wide-field fluorescence microscopy,” Micron 39(7), 808–818 (2008). [CrossRef] [PubMed]

]:

I=(I1I2)2+(I1I3)2+(I2I3)2.
(1)

The axial resolution of depth-resolved IMS (with a 63 × oil immersion objective on the microscope) was measured by imaging sub-resolution point sources – green fluorescent beads (Dia = 175 nm, Invitrogen PS-Speck Microscope Point Source Kit). By scanning the sample in the z axis, the axial PSF was measured at the wavelength 530 nm (see the red line in Fig. 2
Fig. 2 Axial PSF measurement in the depth-resolved IMS.
). To provide a baseline reference, the theoretical axial PSF at this wavelength is also shown as a blue dashed line in Fig. 2. The axial resolution reszis calculated as the full-width-half-maximum (FWHM) of the measured axial PSF, which is:

resz=FWHM(PSF)=1.08 μm .
(2)

This imaging result proves that the depth-resolved IMS has similar sectioning capability as regular wide-field imaging modalities with SI. Although the raw intensity images are remapped and reconstructed in the IMS, it does not reduce the effectiveness of SI in allowing depth-resolved imaging.

3. Multispectral fluorescence microscopic imaging with depth-resolved IMS

To demonstrate the IMS’ sectioning and spectral imaging capabilities in fluorescence microscopy, we imaged a multispectral microsphere (FocalCheck test slide #2, Invitrogen). The diameter of the fluorescent microsphere is 6 μm. Its core and shell are stained with two different fluorescent dyes: The fluorescence emitted from the core has a peak emission at 524 nm; while the fluorescence emitted from the shell has a peak emission at 511 nm (See Fig. 2 (b). The core and shell reference spectra were acquired from single dye stained control sample on the same test slide). A Chroma filter set 41015 (485 LP) was used to separate fluorescence from excitation light.

The spectral imaging results from a sectioned depth layer are shown in Fig. 3
Fig. 3 Spectral imaging of a sectioned depth layer of a fluorescent microsphere by depth-resolved IMS. (a) Panchromatic display of acquired datacube. (b) The spectra of the core and ring of the microsphere. (c) – (d) Pseudo-colored core and ring components after spectral unmixing. (e) A merged image of (c) and (d). (f) Intensity profile of two fluorescent dyes across the microsphere section. (g) Measured spectrum vs. fitted spectrum at the interface of the two fluorophores.
. Due to significant spectral overlaps of the dyes, in the panchromatic display of the acquired datacube (Fig. 3(a)), the core and shell cannot be discriminated. After applying a linear spectral unmixing algorithm [15

15. T. Zimmermann, “Spectral imaging and linear unmixing in light microscopy,” in Advances in Biochemical Engineering Biotechnology, T. Scheper, ed. (Springer, 2005).

] to the acquired datacube, the fluorescent emissions from core (peak at 524 nm) and shell (peak at 511 nm) are successfully separated (Fig. 3(c)–(e)). The spectral unmixing accuracy is evaluated at the pixel level near the separating interface of the two fluorophores. The result shows that the fitted spectrum matches well with the measured spectrum (see Fig. 3 (g)). In addition, a dashed line is drawn across the bead section (Fig. 3(e)) to show the intensity changes of two fluorescent dyes (Fig. 3(f)). No fluorescence emitted from the shell is detected in the core. This demonstrates the sectioning and spectral imaging capabilities of depth-resolved IMS.

Next, we imaged a mouse kidney tissue section with a combination of fluorescent dyes. In the sample, the glomeruli and convoluted tubules were stained with Alexa Fluor 488 (exc: 493 nm, emi: 516 nm), and the filamentous actin was stained with Alexa Fluor 568 (exc: 578 nm, emi: 604 nm). Chroma filter set 61001 (DAPI/FITC/PI) was used to separate fluorescence from excitation light. The integration time for each phase image was 0.1 sec. Selected spectral channel images are shown in Fig. 4 (a)
Fig. 4 Spectral channel images of mouse kidney tissue section acquired by (a) depth-resolved IMS, and (b) IMS under wide-field imaging.
. For comparison, the same FOV was also imaged by the IMS in wide-field imaging and shown in Fig. 4 (b). Comparing Fig. 4 (a) with Fig. 4 (b), the image contrast acquired by the depth-resolved IMS has been significantly increased due to reduced spatial-spectral crosstalk.

To evaluate the 4D (x, y, z, λ) imaging performance of depth-resolved IMS, the mouse kidney tissue section was imaged along the z axis with 1 μm step. The captured 4D (x, y, z, λ) datacube is of size 350 × 350 × 35 × 46, and shown at four selected wavelengths in Fig. 5
Fig. 5 (Media 1 and Media 2) 4D (x, y, z, λ) imaging of a mouse kidney tissue section by the depth-resolved IMS. The glomeruli and convoluted tubules were stained with Alexa Fluor 488 and the filamentous actin was stained with Alexa Fluor 568. The acquired 4D (x, y, z, λ) datacube is of size 350 × 350 × 35 × 46. A total of four out of 46 wavelength (x, y, z) datacubes are shown.
(see Media 1 for the rotation view of (x, y, z) datacubes at four selected wavelengths, and see Media 2 for a scan through of all acquired wavelengths). The x-y cross section views shown in Fig. 5 are taken at the plane with z = 15 μm, and the x-z cross section views are taken at the plane with y = 80 μm. Due to the added sectioning capability, the hollow structure of micro-tubules and filamentous actin is unambiguously revealed by the IMS. This 4D imaging result proves that no artifacts are introduced by integrating traditional 3D (x, y, z) imaging with IMS’ multispectral imaging,

4. Quantitative comparison between the depth-resolved IMS and a hyperspectral confocal microscope

Since the IMS has the sectioning capability due to the implementation of SI, here we quantitatively compare it with a commercially available hyperspectral confocal microscope (HCM) (Zeiss Meta 510) in a depth-resolved fluorescence imaging experiment.

Although the IMS and HCM have similar functionality, technically they behave very differently both at light collection and illumination side. At the light collection side, the IMS is a parallel acquisition system – the spectral information of all spatial sampling points is collected simultaneously in the FOV; while the HCM is a scanning-based system – the spectral information of spatial points is collected in sequence. The HCM’s scanning mechanism causes a severe trade-off between the number of photons collected at each spatial sampling pixel and the whole frame acquisition time – the faster the imager scans, the shorter dwelling time a pixel can have. To some extent, a HCM can compensate this throughput loss at the illumination side by focusing a high-power laser onto the sample. The illumination irradiance in the HCM is usually several orders of magnitude higher than that in the IMS. However, the high illumination irradiance may cause serious photo-damage and photo-toxicity, which is of a particular concern in live cell imaging experiments [16

16. A. Hopt and E. Neher, “Highly nonlinear photodamage in two-photon fluorescence microscopy,” Biophys. J. 80(4), 2029–2036 (2001). [CrossRef] [PubMed]

,17

17. J. Xiao, “Single-molecule imaging in live cells,” in Handbook of Single-Molecule Biophysics, P. Hinterdorfer and A. V. Oijen, eds. (Springer, 2009).

]. Once the fluorophores have been boosted to their saturation excitation state [18

18. J. R. Lakowicz, Principles of fluorescence spectroscopy, 3rd ed. (Springer, 2006).

] – a situation commonly reached by existing confocal systems – even the method of using high power laser falters because the fluorescent emission rate could not be further increased (see the discussion in Section 4.3).

Additionally, the approaches that the IMS and HCM adopt to achieve optical sectioning are also different. Whereas HCM gates the out-of-focus light by a physical pinhole prior to detection, IMS with SI rejects out-of-focus light by computational demodulation after detection. Thus the image noise level acquired under SI is usually higher than that acquired in the HCM.

To provide a quantitative comparison between the IMS and HCM, a standard fluorescent bead imaging experiment was implemented. Since the two imaging modalities use different photon detectors – the IMS uses an interline CCD camera (Imperx, P/N: IPX-16M3-L) while the HCM uses a photomultiplier tube (PMT) array, we first calibrated the photon detector’s gain for each imaging modality (Section 4.1). Then, we measured the maximum number of photons that can be collected from a sectioned depth layer when two imaging modalities were operated at a given frame rate (Section 4.2). The illumination power (Arc lamp for the IMS, Argon laser for the HCM) was maximized in both setups. We also investigated whether the HCM can compensate its scanning-caused throughput loss by using a more powerful laser source in Section 4.3. Since noise amplification occurs in SI measurement, SNR can be a more useful quantity to compare than collected photon count. Thus, we estimated the noise level in the spectral channel images acquired by the depth-resolved IMS and HCM, and compared the two imaging modalities in the context of acquired image SNR (Section 4.4).

4.1 Calibration of photon detector’s gain in the IMS and HCM

For a digital imaging system, the signal Scone acquires from an image is in the unit of counts. To calculate from this quantity the actual number of detected photonsSp, we define the detector’s gain as the conversion relation:

g=photons/count .
(3)

Note that the detector’s gain defined here is not necessarily the same thing as the internal gain value for a PMT, as it includes the amplifier gain and analog-to-digital conversion factor.

For shot noise limited systems, the measured image noise Nc (in counts) in the IMS’ CCD camera and HCM’ PMT are:
Nc,IMS~σc=σp/g,Nc,HCM~1.21/2σc=1.1σp/g,
(4)
where σCis photon noise standard deviation in units of counts, while σpis the photon noise in units of photons. There is a factor 1.21/2 between the measured image noise Nc,HCMand photon noise σcdue to the fact that the PMT is working in the analog mode [19

19. H. Photonics, “Characteristics of Photomultiplier Tubes” in Photomultiplier Tubes: Basics and Applications, 2nd ed. (1999), p. 71.

].

Since photon noise σpobeys Poissonian statistics:

σp2=Sp.
(5)

Combining Eq. (4) and Eq. (5), we have:

Nc,IMS2~Sp/g2=gSc/g2=Sc/g,Nc,HCM2~1.2Sp/g2=1.2gSc/g2=1.2Sc/g.
(6)

Equation (6) indicates that there is a linear relationship between the square of image noise Nc2 (termed signal variance) and image signal levelScwith a slope equal to 1/gin IMS imaging while 1.2/gin HCM imaging.

To experimentally measure the IMS and HCM detector gains, we mounted an integration sphere (FOIS-1, Ocean Optics) on the microscope’s sample stage. The integration sphere was illuminated by a halogen lamp source. The uniform fields were imaged by the IMS and HCM respectively. Specific detector gain settings were chosen on the two modalities based on the criteria that full dynamic range was reached in the subsequent fluorescent bead imaging experiment (Section 4.2). By applying a linear regression to the variables Nc2and Sc at different illumination intensities (see Fig. 6
Fig. 6 Signal variance vs. Intensity in (a) IMS and (b) HCM.
), the IMS and HCM detector’s gains are obtained:

gIMS=17.6 photons/count,gHCM=0.0152 photons/count .
(7)

4.2 Measurement of collected photons in a standard fluorescent bead imaging experiment by the depth-resolved IMS and HCM

In the comparison experiment, a standard green fluorescent bead (Diameter = 2 μm, emi peak: 516 nm) sample was chosen to be imaged by both modalities. Since these fluorescent beads are photo-stable [20

20. The Molecular Probes Handbook, Invitrogen, Inc.

], no photobleaching is expected to be observed in the present study. The comparison experiment concept is shown in Fig. 7
Fig. 7 Experimental design concept for comparison study.
. A spatial depth layer (x, y, λ) of a fluorescent bead is acquired by SI in the depth-resolved IMS, and by pinhole filtering in the HCM. Subsequently, this depth-resolved spatial layer is dispersed and imaged into the spectral channels with nominal wavelengthsλ1, λ2  λn. Since the spectral channel λm=516 nm is close to the peak of bead’s fluorescent emission and maximum of detectors’ quantum efficiency curve, it is chosen as the representative in the following comparisons.

The depth-resolved IMS used a Chroma filter set 41015 (exc: 450/50 nm, emi: 485nm LP) to separate fluorescence from excitation light. The HCM excited the sample with an Argon laser (488 nm), and the fluorescence was collected by the same Zeiss objective (63 × /NA = 1.40) that was used for the IMS. Lambda acquisition mode (8 parallel spectral acquisition channels with spectral bin width ~10 nm) was chosen on the HCM’s operation software. The HCM’s pinhole size was set to be 1.26 Airy disks so that the sectioning thickness (~0.9 μm) was close to that of the depth-resolved IMS. The imaging parameters used in the comparison experiment are summarized in Table 1

Table 1. Imaging Parameters for Depth-resolved IMS and HCM in Comparison Experiment

table-icon
View This Table
.

Note that in the depth-resolved IMS, the pixel exposure time is equal to the frame time due to the parallel acquisition of the IMS (although the illumination intensity is halved due to the grid pattern under SI); while in the HCM, the pixel dwelling time is equal to the division of the frame time to the frame sampling minus scanning time due to the mechanical scanning across the field. To cancel out the differences in frame sampling, spectral bin width (assuming spectral flatness over both bin widths), and sectioning thickness between the two modalities, a signal scaling factor is calculated for the depth-resolved IMS with respect to the HCM by:

L=ΔλHCMΔλIMS×ΔzHCMΔzIMS/MHCM2MIMS2=1.11.
(8)

In Eq. (8), the frame sampling is included in the signal scaling factor because in the HCM imaging one can increase the pixel dwelling time by scanning a smaller area.

The fluorescence emitted by a bead was measured by summing all the pixel intensities (in counts) at the location of the bead. A total of 20 fluorescent beads were measured by the depth-resolved IMS and HCM respectively (see Fig. 8
Fig. 8 (a) Depth-resolved IMS imaging vs. (b) HCM imaging in a standard fluorescent bead imaging experiment. The color bar is in the unit of converted photons from image intensity counts.
). The mean fluorescent signals of beads measured by the depth-resolved IMS are SC,IMS=4.87×103 in counts; while those measured by the HCM are SC,HCM=4.84×104 in counts. By converting the measured signal counts to photons and scaling the signal with factor L, the ratio of photons collected by the depth-resolved IMS to the HCM at the given frame rate is:

r=Sp,IMSLSp,HCM=SC,IMSgIMSLSC,HCMgHCM=4.87×103×17.6×1.114.84×104×0.0152=130.
(9)

4.3 Illumination irradiance in the depth-resolved IMS and HCM

In our microscope setups, the illumination power (Arc lamp for the IMS, Argon laser for the HCM) was maximized for both depth-resolved IMS and HCM. To show the difference between the two modalities at the illumination side, we calculate the illumination irradiance on the microscope stage. In the HCM imaging, the power of the 488 nm excitation laser on the sample plane was measured to be 0.32mW. Assuming the illumination power is focused onto a point-spread-function (PSF), the size of which is ~0.44 μm, the illumination irradiance is approximately:

EHCM~0.32 mW14π×PSF2=0.32 mW14π×(1.22λ/NA)2=0.32 mW14π×(0.44 μm)2=2.14 mW/μm2.
(10)

In the depth-resolved IMS imaging, a total of 40 mW illumination power was measured; however, the illumination area is over the entire microscope objective’s FOV (Dia ~400 μm). Thus the illumination irradiance is approximately

EIMS~40 mW14π×(400 μm)2=3.2 ×104mW/μm2.
(11)

The above results indicate that, at the illumination side, HCM can achieve much higher illumination irradiance than the IMS. This approach is usually adopted by the HCM to compensate its scanning-caused throughput loss at the light collection side. However, the high illumination irradiance may cause serious photodamage and phototoxicity, especially in live cell imaging experiments. Since the generation rate of reactive oxygen species (ROS) – the chemical radical suggested as the main source of photodamage and phototoxicity – has a nonlinear relationship with the illumination irradiance [22

22. R. Dixit and R. Cyr, “Cell damage and reactive oxygen species production induced by fluorescence microscopy: effect on mitosis and guidelines for non-invasive fluorescence microscopy,” Plant J. 36(2), 280–290 (2003). [CrossRef] [PubMed]

], the differences between short exposure of high-intensity light (as in the HCM imaging) and continuous exposure to much lower light level (as in the IMS imaging) may yield important differences in the photodamage and phototoxicity in living cells [21

21. P. J. Shaw, “Comparison of widefield/deconvolution and confocal microscopy for three-dimensional imaging,” in Handbook of Biological Confocal Microscopy, 3rd. ed., J. B. Pawley, ed. (Springer, 2006)

]. This fact makes the IMS better suited than the HCM in live cell imaging applications because the light-induced perturbation is minimized.

To investigate whether the photon collection ratio in Eq. (9) can be reduced if HCM use a more powerful laser source than the current setup, we measured the emissions of a fluorescent bead at different excitation laser powers (see Fig. 9
Fig. 9 Excitation laser power vs. Measured signal in HCM imaging.
). The maximal number of photons that the fluorophore can emit before reaching its saturation state [18

18. J. R. Lakowicz, Principles of fluorescence spectroscopy, 3rd ed. (Springer, 2006).

] is calculated by fitting an exponential curve I=aaebP to the measured data (I is the measured fluorescence signal and P is the excitation laser power). The fitting result shows the decay coefficient b is equal to 2.61mW1. The signal ratio of the fluorophore at the laser power P=toP=0.32 mWis thus obtained:

ξ=I(P=)I(P=0.32 mW)=aa(1e0.32b)=1.77.
(12)

Consequently, if the HCM uses a more powerful laser source in the presented comparison study, the measured photons by the depth-resolved IMS still remains:
r*>r/ξ=73
(13)
times higher than what can be measured by the HCM.

4.4 Noise level estimate and image S/N comparison

Here we consider both systems to be shot noise limited. In HCM imaging, the out-of-focus light is rejected prior to the detection, so the image noise is mainly contributed by the photon noise from the sectioned depth layer. Thus the image SNR for a sectioned fluorescent bead in the HCM imaging is:

SNRHCM=Sp,HCMNp,HCM=Sp,HCM1.21/2σp=Sp,HCM1.2Sp,HCM=Sp,HCM/1.2=24.7.
(14)

In Eq. (14), Sp,HCMis the fluorescent signal from a sectioned bead depth layer and is calculated in Eq. (9).

In the depth-resolved IMS imaging, the out-of-focus light is rejected after detection, so that the image noise is contributed by the photon noise both from in-focus and out-of-focus depth layers. To measure the actual noise in the depth-resolved IMS imaging, we captured 1000 time-sequenced (x, y, z, λ, t) datacubes for the same fluorescent bead sample. Then the mean fluorescent intensity I¯and its standard deviation σIat a bead spatial sampling point (x1,y1,z1)in spectral channel λ = 516 nm are calculated as:

I¯=11000j=11000I(x1,y1,z1,tj,λ=516nm),σI=110001j=11000(I(x1,y1,z1,tj,λ=516nm)I¯)2.
(15)

The measured actual noise σIis 4.78 times higher than the sectioned layer’s photon noiseI¯. Assuming the ratio σI/I¯is the same for all spatial sampling points (x, y, z) at the bead location, the image SNR for a sectioned fluorescent bead in the IMS imaging is estimated to be:
SNRIMS=ijI(xi,yj)ijσI2(xi,yj)=ijI(xi,yj)4.78ijI(xi,yj)=14.78LSp,IMS=64.7,
(16)
where Sp,IMSis the fluorescent signal from a sectioned bead depth layer and is calculated in Eq. (9). L is the signal scaling factor defined in Eq. (8).

By comparing Eq. (14) with Eq. (16), we have following conclusions: at a frame rate of ~1 fps, the image SNR acquire by the depth-resolved IMS is
SNRIMSSNRHCM~2.6
(17)
times higher than that acquired by the hyperspectral confocal approach. Note that this result is also suggested by Fig. 8.

5. Conclusions

In this paper, we present a depth-resolved IMS which is capable of acquiring 4D (x, y, z, λ) datacubes. By implementing structured illumination, the IMS has the same sectioning capability as the regular wide-field imaging modalities but with additional spectral resolving power. The 4D imaging of a mouse kidney tissue section proves that no artifacts are introduced by integrating traditional 3D (x, y, z) imaging with IMS’s multispectral imaging.

SI is an effective way to add the sectioning capability to the IMS, especially due to its easy integration with the IMS on a standard microscope setup. However, under SI the out-of-focus light still uses the detector’s dynamic range and thus contributes to the shot noise, so that the acquired image noise level is usually higher than for HCM, a fact that decreases the advantages of the IMS at the light collection side. However, in the presented study, the acquired image SNR by the depth-resolved IMS is still ~2.6 times higher than that acquired by the HCM. For the imaging applications in which the out-of-focus light dominates the detector’s dynamic range, the alternative implementation of selective plane illumination [24

24. J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, and E. H. Stelzer, “Optical sectioning deep inside live embryos by selective plane illumination microscopy,” Science 305(5686), 1007–1009 (2004). [CrossRef] [PubMed]

] with the IMS may provide better image contrast due to the illumination’s intrinsic sectioning capability.

In addition, in the comparison both IMS and HCM are considered to be shot noise limited because the detectors’ full dynamic range is used. Generally speaking, the readout noise in the CCD camera (e.g., ~16 electrons in the IMS’ camera) is higher than that in the HCM’s PMT (< 1 electron). For low light imaging experiments, the CCD camera’s relatively high readout noise may become a concern. However, in the recently developed scientific CMOS (sCMOS) cameras [25

25. Fairchild, Inc., Andor, Inc., and PCO, Inc., “sCMOS data sheet,” http://www.scmos.com/.

], readout noise has been significantly reduced to less than 2 electrons even at very high frame rate (100 fps). Incorporation of such cameras into the IMS will not only increase IMS’ acquisition speed, but also provide a background noise level comparable to that in HCM imaging.

Acknowledgment

This work is supported by the National Institute of Health under Grant No. R21EB009186 and R21EB011598.

References and links

1.

J. W. Lichtman and J. A. Conchello, “Fluorescence microscopy,” Nat. Methods 2(12), 910–919 (2005). [CrossRef] [PubMed]

2.

Y. Hiraoka, T. Shimi, and T. Haraguchi, “Multispectral imaging fluorescence microscopy for living cells,” Cell Struct. Funct. 27(5), 367–374 (2002). [CrossRef] [PubMed]

3.

T. Zimmermann, J. Rietdorf, and R. Pepperkok, “Spectral imaging and its applications in live cell microscopy,” FEBS Lett. 546(1), 87–92 (2003). [CrossRef] [PubMed]

4.

V. Raicu, M. R. Stoneman, R. Fung, M. Melnichuk, D. B. Jansma, L. F. Pisterzi, S. Rath, M. Fox, J. W. Wells, and D. K. Saldin, “Determination of supramolecular structure and spatial distribution of protein complexes in living cells,” Nat. Photonics 3(2), 107–113 (2009). [CrossRef]

5.

S. Kumazaki, M. Hasegawa, M. Ghoneim, Y. Shimizu, K. Okamoto, M. Nishiyama, H. Oh-Oka, and M. Terazima, “A line-scanning semi-confocal multi-photon fluorescence microscope with a simultaneous broadband spectral acquisition and its application to the study of the thylakoid membrane of a cyanobacterium Anabaena PCC7120,” J. Microsc. 228(Pt 2), 240–254 (2007). [CrossRef] [PubMed]

6.

H. R. Morris, C. C. Hoyt, and P. J. Treado, “Imaging spectrometers for fluorescence and Raman microscopy - acoustooptic and liquid-crystal tunable filters,” Appl. Spectrosc. 48(7), 857–866 (1994). [CrossRef]

7.

B. K. Ford, C. E. Volin, S. M. Murphy, R. M. Lynch, and M. R. Descour, “Computed tomography-based spectral imaging for fluorescence microscopy,” Biophys. J. 80(2), 986–993 (2001). [CrossRef] [PubMed]

8.

C. F. Cull, K. Choi, D. J. Brady, and T. Oliver, “Identification of fluorescent beads using a coded aperture snapshot spectral imager,” Appl. Opt. 49(10), B59–B70 (2010). [CrossRef] [PubMed]

9.

A. Gorman, D. W. Fletcher-Holmes, and A. R. Harvey, “Generalization of the Lyot filter and its application to snapshot spectral imaging,” Opt. Express 18(6), 5602–5608 (2010). [CrossRef] [PubMed]

10.

L. Gao, R. T. Kester, and T. S. Tkaczyk, “Compact image slicing spectrometer (ISS) for hyperspectral fluorescence microscopy,” Opt. Express 17(15), 12293–12308 (2009). [CrossRef] [PubMed]

11.

L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy,” Opt. Express 18(14), 14330–14344 (2010). [CrossRef] [PubMed]

12.

R. T. Kester, N. Bedard, L. Gao, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J. Biomed. Opt. 16(5), 056005 (2011). [CrossRef] [PubMed]

13.

M. A. A. Neil, R. Juskaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22(24), 1905–1907 (1997). [CrossRef] [PubMed]

14.

D. Karadaglić and T. Wilson, “Image formation in structured illumination wide-field fluorescence microscopy,” Micron 39(7), 808–818 (2008). [CrossRef] [PubMed]

15.

T. Zimmermann, “Spectral imaging and linear unmixing in light microscopy,” in Advances in Biochemical Engineering Biotechnology, T. Scheper, ed. (Springer, 2005).

16.

A. Hopt and E. Neher, “Highly nonlinear photodamage in two-photon fluorescence microscopy,” Biophys. J. 80(4), 2029–2036 (2001). [CrossRef] [PubMed]

17.

J. Xiao, “Single-molecule imaging in live cells,” in Handbook of Single-Molecule Biophysics, P. Hinterdorfer and A. V. Oijen, eds. (Springer, 2009).

18.

J. R. Lakowicz, Principles of fluorescence spectroscopy, 3rd ed. (Springer, 2006).

19.

H. Photonics, “Characteristics of Photomultiplier Tubes” in Photomultiplier Tubes: Basics and Applications, 2nd ed. (1999), p. 71.

20.

The Molecular Probes Handbook, Invitrogen, Inc.

21.

P. J. Shaw, “Comparison of widefield/deconvolution and confocal microscopy for three-dimensional imaging,” in Handbook of Biological Confocal Microscopy, 3rd. ed., J. B. Pawley, ed. (Springer, 2006)

22.

R. Dixit and R. Cyr, “Cell damage and reactive oxygen species production induced by fluorescence microscopy: effect on mitosis and guidelines for non-invasive fluorescence microscopy,” Plant J. 36(2), 280–290 (2003). [CrossRef] [PubMed]

23.

Y. Sako, A. Sekihata, Y. Yanagisawa, M. Yamamoto, Y. Shimada, K. Ozaki, and A. Kusumi, “Comparison of two-photon excitation laser scanning microscopy with UV-confocal laser scanning microscopy in three-dimensional calcium imaging using the fluorescence indicator Indo-1,” J. Microsc. 185(Pt 1), 9–20 (1997). [CrossRef] [PubMed]

24.

J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, and E. H. Stelzer, “Optical sectioning deep inside live embryos by selective plane illumination microscopy,” Science 305(5686), 1007–1009 (2004). [CrossRef] [PubMed]

25.

Fairchild, Inc., Andor, Inc., and PCO, Inc., “sCMOS data sheet,” http://www.scmos.com/.

OCIS Codes
(170.6280) Medical optics and biotechnology : Spectroscopy, fluorescence and luminescence
(180.2520) Microscopy : Fluorescence microscopy
(110.4234) Imaging systems : Multispectral and hyperspectral imaging

ToC Category:
Medical Optics and Biotechnology

History
Original Manuscript: July 11, 2011
Revised Manuscript: August 17, 2011
Manuscript Accepted: August 18, 2011
Published: August 19, 2011

Virtual Issues
Vol. 6, Iss. 9 Virtual Journal for Biomedical Optics

Citation
Liang Gao, Noah Bedard, Nathan Hagen, Robert T. Kester, and Tomasz S. Tkaczyk, "Depth-resolved image mapping spectrometer (IMS) with structured illumination," Opt. Express 19, 17439-17452 (2011)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-19-18-17439


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. J. W. Lichtman and J. A. Conchello, “Fluorescence microscopy,” Nat. Methods2(12), 910–919 (2005). [CrossRef] [PubMed]
  2. Y. Hiraoka, T. Shimi, and T. Haraguchi, “Multispectral imaging fluorescence microscopy for living cells,” Cell Struct. Funct.27(5), 367–374 (2002). [CrossRef] [PubMed]
  3. T. Zimmermann, J. Rietdorf, and R. Pepperkok, “Spectral imaging and its applications in live cell microscopy,” FEBS Lett.546(1), 87–92 (2003). [CrossRef] [PubMed]
  4. V. Raicu, M. R. Stoneman, R. Fung, M. Melnichuk, D. B. Jansma, L. F. Pisterzi, S. Rath, M. Fox, J. W. Wells, and D. K. Saldin, “Determination of supramolecular structure and spatial distribution of protein complexes in living cells,” Nat. Photonics3(2), 107–113 (2009). [CrossRef]
  5. S. Kumazaki, M. Hasegawa, M. Ghoneim, Y. Shimizu, K. Okamoto, M. Nishiyama, H. Oh-Oka, and M. Terazima, “A line-scanning semi-confocal multi-photon fluorescence microscope with a simultaneous broadband spectral acquisition and its application to the study of the thylakoid membrane of a cyanobacterium Anabaena PCC7120,” J. Microsc.228(Pt 2), 240–254 (2007). [CrossRef] [PubMed]
  6. H. R. Morris, C. C. Hoyt, and P. J. Treado, “Imaging spectrometers for fluorescence and Raman microscopy - acoustooptic and liquid-crystal tunable filters,” Appl. Spectrosc.48(7), 857–866 (1994). [CrossRef]
  7. B. K. Ford, C. E. Volin, S. M. Murphy, R. M. Lynch, and M. R. Descour, “Computed tomography-based spectral imaging for fluorescence microscopy,” Biophys. J.80(2), 986–993 (2001). [CrossRef] [PubMed]
  8. C. F. Cull, K. Choi, D. J. Brady, and T. Oliver, “Identification of fluorescent beads using a coded aperture snapshot spectral imager,” Appl. Opt.49(10), B59–B70 (2010). [CrossRef] [PubMed]
  9. A. Gorman, D. W. Fletcher-Holmes, and A. R. Harvey, “Generalization of the Lyot filter and its application to snapshot spectral imaging,” Opt. Express18(6), 5602–5608 (2010). [CrossRef] [PubMed]
  10. L. Gao, R. T. Kester, and T. S. Tkaczyk, “Compact image slicing spectrometer (ISS) for hyperspectral fluorescence microscopy,” Opt. Express17(15), 12293–12308 (2009). [CrossRef] [PubMed]
  11. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy,” Opt. Express18(14), 14330–14344 (2010). [CrossRef] [PubMed]
  12. R. T. Kester, N. Bedard, L. Gao, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J. Biomed. Opt.16(5), 056005 (2011). [CrossRef] [PubMed]
  13. M. A. A. Neil, R. Juskaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett.22(24), 1905–1907 (1997). [CrossRef] [PubMed]
  14. D. Karadaglić and T. Wilson, “Image formation in structured illumination wide-field fluorescence microscopy,” Micron39(7), 808–818 (2008). [CrossRef] [PubMed]
  15. T. Zimmermann, “Spectral imaging and linear unmixing in light microscopy,” in Advances in Biochemical Engineering Biotechnology, T. Scheper, ed. (Springer, 2005).
  16. A. Hopt and E. Neher, “Highly nonlinear photodamage in two-photon fluorescence microscopy,” Biophys. J.80(4), 2029–2036 (2001). [CrossRef] [PubMed]
  17. J. Xiao, “Single-molecule imaging in live cells,” in Handbook of Single-Molecule Biophysics, P. Hinterdorfer and A. V. Oijen, eds. (Springer, 2009).
  18. J. R. Lakowicz, Principles of fluorescence spectroscopy, 3rd ed. (Springer, 2006).
  19. H. Photonics, “Characteristics of Photomultiplier Tubes” in Photomultiplier Tubes: Basics and Applications, 2nd ed. (1999), p. 71.
  20. The Molecular Probes Handbook, Invitrogen, Inc.
  21. P. J. Shaw, “Comparison of widefield/deconvolution and confocal microscopy for three-dimensional imaging,” in Handbook of Biological Confocal Microscopy, 3rd. ed., J. B. Pawley, ed. (Springer, 2006)
  22. R. Dixit and R. Cyr, “Cell damage and reactive oxygen species production induced by fluorescence microscopy: effect on mitosis and guidelines for non-invasive fluorescence microscopy,” Plant J.36(2), 280–290 (2003). [CrossRef] [PubMed]
  23. Y. Sako, A. Sekihata, Y. Yanagisawa, M. Yamamoto, Y. Shimada, K. Ozaki, and A. Kusumi, “Comparison of two-photon excitation laser scanning microscopy with UV-confocal laser scanning microscopy in three-dimensional calcium imaging using the fluorescence indicator Indo-1,” J. Microsc.185(Pt 1), 9–20 (1997). [CrossRef] [PubMed]
  24. J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, and E. H. Stelzer, “Optical sectioning deep inside live embryos by selective plane illumination microscopy,” Science305(5686), 1007–1009 (2004). [CrossRef] [PubMed]
  25. Fairchild, Inc., Andor, Inc., and PCO, Inc., “sCMOS data sheet,” http://www.scmos.com/ .

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Supplementary Material


» Media 1: MOV (2294 KB)     
» Media 2: MOV (2010 KB)     

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited