Analysis on enhanced depth of field for integral imaging microscope

doi:10.1364/OE.20.023480

« Show journal navigation

Analysis on enhanced depth of field for integral imaging microscope

Young-Tae Lim, Jae-Hyeung Park, Ki-Chul Kwon, and Nam Kim »View Author Affiliations

Optics Express, Vol. 20, Issue 21, pp. 23480-23488 (2012)
http://dx.doi.org/10.1364/OE.20.023480

View Full Text Article

Acrobat PDF (1228 KB)

Journal Search
Article Lookup

Article Tools

Citations

Alert me when this article is cited
Export Citation/Save

Article Navigation

▸ Article Outline
– Abstract
1. Introduction
2. Lateral resolution in integral imaging microscope
3. DOF in IIM
4. Experiments for generating extended DOF images and comparison with conventional extended depth of field images
5. Conclusion
– References and links

Article Tools
Full Text
Article Info
References
Cited By
Figures
Metrics
Download PDF

Viewing Equations in the
HTML Article

Optimal Viewing Recommendations

Full Text
Article Info
References (11)
Cited By
Figures (11)
Metrics

Abstract

Depth of field of the integral imaging microscope is studied. In the integral imaging microscope, 3-D information is encoded as a form of elemental images Distance between intermediate plane and object point decides the number of elemental image and depth of field of integral imaging microscope. From the analysis, it is found that depth of field of the reconstructed depth plane image by computational integral imaging reconstruction is longer than depth of field of optical microscope. From analyzed relationship, experiment using integral imaging microscopy and conventional microscopy is also performed to confirm enhanced depth of field of integral imaging microscopy.

1. Introduction

Three dimensional (3-D) information capture techniques including pickup, display, and image processing have been widely studied in the fields of science, industry, the military, broadcasting, and bio-medicine. Especially, 3-D information capture techniques of microscopic objects have become a main concern of microscopy. The objective lens of the optical microscope has a shallow depth of field (DOF) so that a thin section of the object can be imaged [1

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13(1), 191–219 (1984). [CrossRef] [PubMed]

–5

C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35(7), 1169–1185 (1988). [CrossRef]

]. Hence, a stack of focal images are required to reconstruct the 3-D information of an object [6

S. Inoue and R. Oldenbourg, Handbook of Optics (McGrawHill, 1995), Chap. 17.

Q. Wu, F. A. Merchant, and K. R. Castleman, Microscope Image Processing (Academic press, 2008), Chap. 2.

]. A representative 3-D information capture technique of microscopy is confocal microscopy. Confocal microscopy is a technique whereby a researcher captures different depth images of an object by axial scanning. Then these image sets are synthesized in depth plane order. A synthesized depth plane image set has extended DOF. Another common technique of 3-D microscopy is digital holographic microscopy (DHM) [8

J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt. 45(16), 3893–3901 (2006). [CrossRef] [PubMed]

]. DHM offers full volumetric information of the object by numerical processing. However, it has been limited by difficulties in practical use, such as those encountered when capturing full color images and requiring coherent illumination. Integral imaging microscopy is the 3-D microscopy that provides different depth plane information; with it you view images and full-color information of the object by numerical processing. An integral imaging microscope (IIM) acquires 3-D information in an elemental form using a micro lens array. Due to the micro lens array, the DOF of the IIM can be extended to capture a thick object. Acquired 3-D information, i.e. elemental images, of the object is encoded as a form of the disparity between the elemental images, and simple pixel mapping methods are used to generate views and depth plane images. Thus, generated depth plane images are free from off-axis distortion caused by mechanical movements [9

Y.-T. Lim, J.-H. Park, N. Kim, and K.-C. Kwon, “Dense light field microscopy,” Proc. SPIE 7237, 72371Q, 72371Q-12 (2009). [CrossRef]

,10

Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009). [CrossRef] [PubMed]

In this paper, we analyze the extended DOF of the IIM compared with conventional optical microscopes. Each depth plane image reconstructed by the computational integral imaging reconstruction method (CIIR) in the IIM is compared to the sequentially-captured 2-D images by a conventional optical microscope. Also, profiles of the reconstructed depth plane images are generated to verify efficient 3-D visualization. In Section 2, the principle of lateral resolution in the IIM is explained. In Section 3, capturing elemental image conditions related with the DOF of the IIM and simulation results are explained and presented. In Section 4, the experimental results of the DOF image comparison between the IIM and conventional optical microscope are presented. In section 5, we explain our conclusions.

2. Lateral resolution in integral imaging microscope

In conventional IIM, orthographic view image reconstruction (OVIR) and the CIIR method are used to generate multiple view images and different depth plane images [11

D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor. 12(3), 131–135 (2008). [CrossRef]

]. To analyze the DOF in IIM, it is necessary to consider two factors, lateral and axial resolution of the elemental image and the numerical aperture (NA) of the micro lens array. The 3-D information including different views and depth planes is encoded as a disparity between the elemental images. Hence, the axial resolution is related to the lateral resolution.

The IIM is composed of the objective lens, tube lens, micro lens array, and charge coupled device (CCD). The structure of the IIM is shown in Fig. 1 .

Fig. 1 Structure of the integral imaging microscope.

In Fig. 1, f₁ and f₂ are the focal length of the objective lens and tube lens, respectively; d is the distance between the intermediate plane to the micro lens array; g is the gap between the micro lens array and the CCD sensor; φ is the pitch size of the micro lens array; and e is the pixel size of the CCD. In the conventional optical microscope, lateral resolution is given by

R = \frac{0.47 λ}{N A},

(1)

where R is lateral resolution, λ is wavelength, and NA is the NA of the objective lens. In the object plane of the IIM, the minimum resolvable spot is same as a conventional optical microscope in Eq. (1). In the CCD sensor plane, lateral resolution is related to magnification of the optical microscope, g, d, and NA of the micro lens array. The minimum resolvable spot in the object plane is given by

R = \frac{0.47 λ}{M \cdot N A_{l a}},

(2)

where M is the magnification of the optical microscope and NA_la is the NA of the micro les array. From Eq. (2), the minimum resolvable spot in the intermediate plane is derived by

R_{int} = M R = \frac{0.47 λ}{N A} M .

(3)

From Eq. (2) and Eq. (3), the minimum resolvable spot in the elemental image plane is derived by

R_{e l e} = \frac{0.47 λ}{N A_{l a}} \cdot \frac{g}{d} .

(4)

If the R_ele is bigger than the pixel pitch of the CCD, a blurred image will be captured. Resolution of the reconstructed view images and depth plane images are also blurred. There are three types of imaging cases of IIM. First, if the NA of the micro lens array, NA_la, is the same as NA/M, all rays from the objective lens are collected by the micro lens array. In that case, loss of resolution does not occur due to the micro lens array. In this case, each object point is imaged to only one or two elemental lenses. Hence, the reconstructed view image has one or two views. Second, NA_la is bigger than NA/M. In this case, each object point is imaged to only one or two elemental lenses, but the partial aperture of the micro lens array is used. Third, NA_la is smaller than NA/M. In this case, some rays from the objective lens are not collected by a micro lens. Captured elemental image set by CCD is limited by number of elemental lens. In this case, total pixel count of reconstructed view images and depth plane images by using OVIR and CIIR method are limited by NA_la. This tradeoff is traditional disadvantage of the integral imaging, however, it is necessary to generate 3-D information of the object. Conditions to form an elemental image are shown in Fig. 2 .

Fig. 2 Relationship between numerical aperture of the microscope lens array and intermediate plane when forming an elemental image.

In Fig. 2 f_la is the focal length of the micro lens array. To capture more views of the object, third case has to be satisfied.

3. DOF in IIM

Axial resolution is often related to DOF in the microscope. To explain DOF in a microscope, there are two formulas: geometrical optics and wave optics, both shown in Eq. (5).

D_{o} = D_{w a v e} + D_{g e o m} = \frac{λ n}{N A^{2}} + \frac{n}{M \cdot N A} e,

(5)

where D_o is the total DOF of the conventional optical microscope, D_wave is the DOF with wave optics, D_geom is DOF with geometrical optics, and e is the pixel size of the CCD. As mentioned above, NA of the single micro lens, NA_la, is determined by d and g as shown in Fig. 3 .

Fig. 3 Depth of field in an integral imaging microscope.

DOF by lens array around the intermediate image plane can be express as

D_{l a} = \frac{λ}{N A_{l a}^{2}} + \frac{d}{g \cdot N A_{l a}} e .

(6)

It is magnified as 1/M² by the objective and tube lenses. DOF around the object plane is given by

D = \frac{1}{M^{2}} D_{l a} = \frac{λ}{M^{2} \cdot N A_{l a}^{2}} + \frac{d}{M^{2} \cdot g \cdot N A_{l a}} e .

(7)

From Eq. (4) and Eq. (7), both lateral and axial resolutions are determined by NA_la. Note that extended DOF of the IIM compared to conventional optical microscope means that area of focused depth plane image is extended by micro lens array. The resolution of reconstructed images in the extended DOF has low quality than conventional optical microscope. The IIM still has resolution tradeoff by micro lens array [10

Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009). [CrossRef] [PubMed]

Figure 4 shows simulation results of the calculated DOF of a conventional optical microscope and IIM from Eq. (5) and Eq. (7), where NA = 0.2, λ = 532nm, M = 10, f_la = 2.4mm, e = 6μm, and φ = 125μm. In Fig. 4(a), DOF of the IIM is longer than DOF of the conventional optical microscope when d = 3.3mm. As mentioned above in section 2, NA_la<NA/M, to form an elemental image with multiple view information, d is longer than 3.125mm. In the IIM, the depth information of the object is encoded as a disparity between the elemental images. Hence, the minimum depth deviation which causes a barely detectable change of the disparity between the elemental images can be regarded as a measure of axial resolution. In Fig. 4(b), if d is 3.5mm, then g is 7.6mm. Numerically calculated DOF of the IIM exceed DOF of the conventional optical microscope. In this condition, R_ele is 30.5μm and each object point is imaged to four elemental lens. Four view images can be reconstructed via the OVIR method. However, these images are blurred. R_ele is larger than pixel size of the CCD e. Hence, effective range of d has to be set.

Fig. 4 Simulation result: (a) comparison of DOF with a conventional optical microscope and integral imaging microscope; (b) calculated DOF after changing d and g.

The relationship between the DOF of the IIM and a conventional optical microscope related with Eq. (5) and Eq. (7) is shown in Fig. 5 . The IIM captures a wide range of objects, then the CIIR method separates each depth slice image computationally. Hence, the DOF of the conventional optical microscope is included in the DOF of the IIM.

Fig. 5 DOF region comparison.

4. Experiments for generating extended DOF images and comparison with conventional extended depth of field images

The experimental setup was composed of the Olympus BX-51 and a micro lens array module as shown in Fig. 6(a) . The micro lens array module is composed of the micro lens array, focal lens, and CCD. Each part of micro lens array module can be separable as shown in Fig. 6(b). The NA of the objective lens was 0.2, the wavelength was 532nm, the magnification of the objective lens was 10, the focal length of the micro lens array was 2.4mm, the pixel pitch size of the CCD was 6μm, the pitch size of the micro lens was 125μm and d is 10mm.

Fig. 6 Experimental setup: (a) captured experimental setup; (b) diagram of the micro lens array module.

Figure 7(a) shows the 2D image of the object used in the experiment. The micro register height is about 140μm. From Eq. (7), calculated D, D_o, and R_ele are 180μm, 18μm, and 12.6μm, respectively. Figure 7(b) shows the elemental image of Fig. 7(a). The pixel size and number of pixels of the elemental image are 946 × 946 and 11 × 11, respectively. Calculated DOF of the IIM is 18μm, and then 8 images are required to compare the DOF of a conventional optical microscope shown as Fig. 8 . From Fig. 7(b), depth slice images were generated at a reconstructed distance with z = −3mm to −6mm, and 3mm to 6mm, shown as Fig. 9 . In conventional CIIR method, for a given depth plane, specific depth plane can be generated. CIIR method used in this paper is only effective within integer depth plane . If g is same to focal length of the micro lens array, errors occurs for coding. Each local position of pixels is remapped on certain depth plane. In this case, focal length of the micro lens array is 2.4mm, minimum condition to generate depth plane of center region of the object is 3mm vice versa. In our experiment, 3mm to 6mm and −3mm to −6mm on depth plane of CIIR cover the range of 140μm.

Fig. 7 (a) 2-D micro scale object image and (b) elemental image.

Fig. 8 Sequentially-captured 2D microscopic image of Fig. 7(a).

Fig. 9 Depth plane image generated from Fig. 7(b) using CIIR.

In order to verify that the reconstructed images of IIM are equivalent to the 2-D images of conventional microscopy, we first compared PSNR of Figs. 8 and 9. The average PSNR difference between the two images is 11.75dB, and the maximum difference is 2.2dB, shown as Fig. 10 . Reconstructed depth plane images using CIIR has low resolution and noise which is caused by a boundary between each lens of the micro lens array. Results show that depth plane images using the CIIR method have similar DOF with sequentially-captured 2-D images of conventional optical microscopes. Second, we analyzed image intensities both of Fig. 8 and Fig. 9. Distribution of maximum pixel intensity of Fig. 11(a) and Fig. 11(b) are 95th, 205th, and 380th, and of Fig. 11(c) and Fig. 11(d) are 89th, 218th, and 376th. The difference of maximum intensity between both images is about 40. It is also affected by boundary distortion of the micro lens array while CIIR is processing. Results show that two image sets have a similar profile.

Fig. 10 PSNR of each depth plane image between 2-D and IIM.

Fig. 11 Intensities of pixels: (a) z = 3 to 6mm by CIIR, (b) z = 16.6 to 66.4μm by 2-D, (c) z = −3 to −6mm by CIIR, (d) z = −16.6 to −66.4μm by 2-D.

5. Conclusion

IIM can form elemental images in a unique way. The relationship between lateral resolution and DOF was analyzed to explain the capability to surface reconstructed depth plane images of an object. Corresponding distance from the intermediate plane to the micro lens array and object point around the focal plane of the microscope are main factors in forming an elemental image. These factors determine the disparity, that is, 3-D information of the object. The reconstructed depth plane image has low resolution, but it includes the DOF of the conventional optical microscope. In future work, a resolution-enhancing method such as an intermediate view reconstruction technique will be used for our research. It will be a useful device to acquire 3-D information of the microscope object without mechanical movements.

Acknowledgments

This research was supported by the Leaders in INdustry-university Cooperation(LINC) Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education, Science and Technology (2012-B-0013-010116).

References and links

1.	D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng. 13(1), 191–219 (1984). [CrossRef] [PubMed]
2.	J. G. McNally, C. Preza, J. A. Conchello, and L. J. Thomas Jr., “Artifacts in computational optical-sectioning microscopy,” J. Opt. Soc. Am. A 11(3), 1056–1067 (1994). [CrossRef] [PubMed]
3.	S. Liu and H. Hua, “Extended depth-of-field microscopic imaging with a variable focus microscope objective,” Opt. Express 19(1), 353–362 (2011). [CrossRef] [PubMed]
4.	J. A. Conchello and J. W. Lichtman, “Optical sectioning microscopy,” Nat. Methods 2(12), 920–931 (2005). [CrossRef] [PubMed]
5.	C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt. 35(7), 1169–1185 (1988). [CrossRef]
6.	S. Inoue and R. Oldenbourg, Handbook of Optics (McGrawHill, 1995), Chap. 17.
7.	Q. Wu, F. A. Merchant, and K. R. Castleman, Microscope Image Processing (Academic press, 2008), Chap. 2.
8.	J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt. 45(16), 3893–3901 (2006). [CrossRef] [PubMed]
9.	Y.-T. Lim, J.-H. Park, N. Kim, and K.-C. Kwon, “Dense light field microscopy,” Proc. SPIE 7237, 72371Q, 72371Q-12 (2009). [CrossRef]
10.	Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009). [CrossRef] [PubMed]
11.	D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor. 12(3), 131–135 (2008). [CrossRef]

OCIS Codes
(100.6890) Image processing : Three-dimensional image processing
(110.0180) Imaging systems : Microscopy
(170.6900) Medical optics and biotechnology : Three-dimensional microscopy

ToC Category:
Microscopy

History
Original Manuscript: July 6, 2012
Revised Manuscript: September 1, 2012
Manuscript Accepted: September 24, 2012
Published: September 27, 2012

Citation
Young-Tae Lim, Jae-Hyeung Park, Ki-Chul Kwon, and Nam Kim, "Analysis on enhanced depth of field for integral imaging microscope," Opt. Express 20, 23480-23488 (2012)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-20-21-23480

Sort: Author | Year | Journal | Reset

References

D. A. Agard, “Optical sectioning microscopy: cellular architecture in three dimensions,” Annu. Rev. Biophys. Bioeng.13(1), 191–219 (1984). [CrossRef] [PubMed]
J. G. McNally, C. Preza, J. A. Conchello, and L. J. Thomas., “Artifacts in computational optical-sectioning microscopy,” J. Opt. Soc. Am. A11(3), 1056–1067 (1994). [CrossRef] [PubMed]
S. Liu and H. Hua, “Extended depth-of-field microscopic imaging with a variable focus microscope objective,” Opt. Express19(1), 353–362 (2011). [CrossRef] [PubMed]
J. A. Conchello and J. W. Lichtman, “Optical sectioning microscopy,” Nat. Methods2(12), 920–931 (2005). [CrossRef] [PubMed]
C. J. R. Sheppard and X. Q. Mao, “Confocal microscopes with slit apertures,” J. Mod. Opt.35(7), 1169–1185 (1988). [CrossRef]
S. Inoue and R. Oldenbourg, Handbook of Optics (McGrawHill, 1995), Chap. 17.
Q. Wu, F. A. Merchant, and K. R. Castleman, Microscope Image Processing (Academic press, 2008), Chap. 2.
J. Sheng, E. Malkiel, and J. Katz, “Digital holographic microscope for measuring three-dimensional particle distributions and motions,” Appl. Opt.45(16), 3893–3901 (2006). [CrossRef] [PubMed]
Y.-T. Lim, J.-H. Park, N. Kim, and K.-C. Kwon, “Dense light field microscopy,” Proc. SPIE7237, 72371Q, 72371Q-12 (2009). [CrossRef]
Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express17(21), 19253–19263 (2009). [CrossRef] [PubMed]
D.-H. Shin and E.-S. Kim, “Computational integral imaging reconstruction of 3D Object using a depth conversion technique,” J. Opt. Soc. Kor.12(3), 131–135 (2008). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

Click here to see a list of articles that cite this paper