## Depth and lateral size control of three-dimensional images in projection integral imaging

Optics Express, Vol. 12, Issue 16, pp. 3778-3790 (2004)

http://dx.doi.org/10.1364/OPEX.12.003778

Acrobat PDF (1635 KB)

### Abstract

We present a simple method to pick up (sense) large objects that are far away, and then display their three-dimensional images within the depth of focus of projection integral imaging systems. For this purpose, we propose to use either curved pickup devices or curved display devices or both. In this method, as the object distance increases, the longitudinal image depth reduces in a nonlinear way, while the lateral size reduces in a linear way. To reduce the depth of reconstructed images alone, a method to zoom in elemental images can be used. We analyze the two methods when they are used together. Experiments are presented to show the feasibility of our approach.

© 2004 Optical Society of America

## 1. Introduction

3. A. R. L. Travis, “The display of Three-dimensional video images,” Proc. of IEEE **85**, 1817–1832 (1997). [CrossRef]

4. D. H. McMahon and H. J. Caulfield, “A technique for producing wide-angle holographic displays,” Appl. Opt. **9**, 91–96 (1970). [CrossRef] [PubMed]

12. H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A **15**, 2059–2065 (1998). [CrossRef]

15. S.-W. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. **42**, 4186–4195 (2003). [CrossRef] [PubMed]

8. H. E. Ives, “Optical properties of a Lippmann lenticulated sheet,” J. Opt. Soc. Am. **21**, 171–176 (1931). [CrossRef]

13. J.-S. Jang, F. Jin, and B. Javidi, “Three-dimensional integral imaging with large depth of focus using real and virtual image fields,” Opt. Lett. **28**, 1421–1423 (2003). [CrossRef] [PubMed]

18. J.-S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging using lenslets with non-uniform focal lengths and aperture sizes,” Opt. Lett. **28**, 1924–1926 (2003). [CrossRef] [PubMed]

19. J.-S. Jang, Y.-S. Oh, and B. Javidi, “Spatiotemporally multiplexed integral imaging projector for large-scale high- resolution three-dimensional display,” Opt. Express **12**, 557–563 (2004), http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-4-557. [CrossRef] [PubMed]

20. J.-S. Jang and B. Javidi, “Three-dimensional projection integral imaging using micro-convex-mirror arrays,” Opt. Express **12**, 1077–1083 (2004), http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-6-1077. [CrossRef] [PubMed]

15. S.-W. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. **42**, 4186–4195 (2003). [CrossRef] [PubMed]

17. Y. Kim, J. Park, H. Choi, S. Jung, S. Min, and B. Lee, “Viewing-angle-enhanced integral imaging system using a curved lens array,” Opt. Express **12**, 421–429 (2004), http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-3-421. [CrossRef] [PubMed]

## 2. Review of integral imaging

### 2.1 Conventional integral imaging

*f*is positioned at

*z*=0, and the display panel at

*z*=-

*g*. From the Gauss lens law

*g*should be

*L*

_{i}

*f*/(

*L*

_{i}-

*f*)≡

*g*

_{r}, where it is assumed that 3-D real images are formed around

*z*=

*L*

_{i}. The rays coming from elemental images converge to form a 3-D real image through the lenslet array. The reconstructed 3-D image is a pseudoscopic (depth-reversed) real image of the object. To convert the pseudoscopic image to an orthoscopic image, a process to rotate every elemental image by 180 degrees around its own center optic axis may be used [16

16. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. **37**, 2034–2045 (1998). [CrossRef]

*z*=-

*L*

_{i}, the gap distance

*g*should be

*L*

_{i}

*f*/(

*L*

_{i}+

*f*)≡

*g*

_{v}for optimal focusing from Eq. (1).

## 2.2 Projection integral imaging (PII)

*z*=-

*g*

_{r}, as depicted in Fig. 2(a) and (c). If we use P/O-converted elemental images to display 3-D orthoscopic virtual images, which are formed around

*z*=-

*L*

_{i}, the in-focus plane of projected elemental images should be positioned at

*z*=-

*g*

_{v}.

20. J.-S. Jang and B. Javidi, “Three-dimensional projection integral imaging using micro-convex-mirror arrays,” Opt. Express **12**, 1077–1083 (2004), http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-6-1077. [CrossRef] [PubMed]

*z*=-

*L*

_{i}and the focal length of the lenslet array (or the micro-convex-mirror array) is -

*f*. Then, the gap distance

*g*becomes

*L*

_{i}

*f*/(

*f*-

*L*

_{i})≡-

*g*

_{r}from Eq. (1). Thus the in-focus plane of projected elemental images should be positioned at

*z*=+

*g*

_{r}, as depicted in Fig. 2(b) and (d). On the other hand, when we display 3-D real images around

*z*=

*L*

_{i}, the in-focus plane of projected elemental images should be positioned at

*z*=+

*g*

_{v}. Because

*L*

_{i}≫

*f*in both PII and CII,

*g*

_{r}≈

*g*

_{v}≈

*f*.

## 2.3 Advantages of PII over CII

*ψ*is limited and determined approximately by 2×arctan[0.5/(

*f*/#)], where

*f*/# is the

*f*number of the lenslet, when the fill factor of the lenslet array is close to 1 [14

14. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging with nonstationary micro-optics,” Opt. Lett. **27**, 324–326 (2002). [CrossRef]

16. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. **37**, 2034–2045 (1998). [CrossRef]

*f*/# than it is to make similar lenslets. Each convex mirror element could have an

*f*/# smaller than 1. For example, if

*f*/#=0.5, the viewing angle

*ψ*becomes 90 degrees, which is acceptable for many practical applications.

19. J.-S. Jang, Y.-S. Oh, and B. Javidi, “Spatiotemporally multiplexed integral imaging projector for large-scale high- resolution three-dimensional display,” Opt. Express **12**, 557–563 (2004), http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-4-557. [CrossRef] [PubMed]

## 3. Longitudinal depth control of 3-D images

*δ*. It was shown that

*δ*cannot be larger than 1/(

*λρ*

^{2}) where

*λ*is the display wavelength and

*ρ*is the resolution of reconstructed 3-D images [13

13. J.-S. Jang, F. Jin, and B. Javidi, “Three-dimensional integral imaging with large depth of focus using real and virtual image fields,” Opt. Lett. **28**, 1421–1423 (2003). [CrossRef] [PubMed]

18. J.-S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging using lenslets with non-uniform focal lengths and aperture sizes,” Opt. Lett. **28**, 1924–1926 (2003). [CrossRef] [PubMed]

*ρ*is defined as the inverse of the reconstructed image spot size. In PII, 3-D images with high resolution can be reconstructed only near the projection screen of micro-convex-mirror arrays (or the display lenslet array). Thus the depth-of-focus

*δ*should be measured from the projection screen.

*T*, is positioned at

*z*=

*z*

_{o}>

*δ*. When the focal lengths of the pickup lenslets and the micro-convex-mirrors in the projection screen are equal in magnitude, a 3-D image is reconstructed either at

*z*=

*z*

_{o}for real image display or at

*z*=-

*z*

_{o}for virtual image display. Then, we cannot display a focused 3-D image because the image position is beyond the range of depth-of-focus. Therefore, we need to control the depth (and thus position) of reconstructed 3-D integral images to be displayed so that it can be reconstructed near the screen, i.e., within the depth-of-focus.

## 3.1 Linear depth control by zooming the elemental images

*f*

_{p}is longer than that of the display micro-convex-mirror array

*f*

_{d}, the longitudinal scale of reconstructed image space is reduced linearly by a factor of

*f*

_{d}/

*f*

_{p}≡

*r*while the lateral scale does not change. So if (

*z*

_{o}+

*T*)

*r*<

*δ*, the 3-D reconstructed image is well focused.

*f*

_{p}or an array of micro-zoom lenses. If we increase

*f*

_{p}by a factor of

*α*, every elemental image is also magnified by that factor, according to geometrical optics. Therefore, digital zoom-in can be used, even if

*f*

_{p}is fixed. In other words, by digitally magnifying every elemental images in a computer by a factor of

*α*, we can change

*r*as

*z*=-

*rz*

_{o}for the object positioned at

*z*=

*z*

_{o}in the pickup process.

*z*

_{o}→∞ and the object is very large, this linear control method cannot be used. So, we will consider a nonlinear depth control method.

## 3.2 Nonlinear depth control using curved pickup devices

*R*, and then reconstruct 3-D images using planar display devices as depicted in Fig. 3(a) and (b), respectively. Similarly, planar pickup devices and curved display devices may be used as depicted in Fig. 3(c) and (d), respectively. We use the following sign convention:

*R*>0, when the center of the curvature is positioned at the same side of the object (observer) in the pickup (display) process; and

*R*<0 when it is positioned at the opposite side.

*R*, 3-D images of large objects that are far away can be displayed within the depth-of-focus of the II system.

*R*

_{p}, which is in contact with the planar pickup lenslet array, as depicted in Fig. 3(e). This is because ray propagation behaviors for the two setups in Fig. 3(a) and 3(e), and those in Fig. 3(d) and 3(f) are the same, respectively. We call this lens an optical path-length-equalizing (OPLE) lens. When two thin lenses with focal length

*f*

_{1}and

*f*

_{2}are in contact, the effective focal length becomes

*f*

_{1}

*f*

_{2}/(

*f*

_{1}+

*f*

_{2}). To get complete equivalence between the two setups, the focal length of the lenslet array that is in contact with the OPLE lens should be

*R*

_{p}

*f*

_{p}/(

*R*

_{p}+

*f*

_{p}), where

*f*

_{p}is the focal length of the curved pickup lenslets. In general,

*R*

_{p}≫

*f*

_{p}and thus

*f*

_{p}. Therefore, instead of using the curved pickup lenslet array with a radius of curvature -

*R*

_{p}and a focal length

*f*

_{p}, and a curved image sensor in the analysis, we can use a planar lenslet array with a focal length

*R*

_{p}.

*z*=

*z*

_{o}(>0), the OPLE lens produces its image according to Eq. (1) at

*z*

_{o}varies from ∞ to 0,

*z*

_{i}changes from

*R*

_{p}to 0. The elemental images with increased disparity are projected onto a planar micro-convex-mirror array screen, a virtual image is reconstructed at

*z*=-

*z*

_{i}if

*f*

_{d}=

*f*

_{p}. Therefore,

*R*

_{p}should be shorter than the depth-of-focus of the II system. Lateral magnification of the OPLE lens is given by

*z*

_{i}/

*z*

_{o}(<1) according to geometrical optics.

*R*

_{d}are used, while elemental images are obtained by use of planar pickup devices. As before, we introduce a hypothetical display OPLE lens to planar display devices. Then, an orthoscopic virtual image of the object is reconstructed at

*z*=

*z*

_{o}(>0) in the pickup process, if

*f*

_{d}=

*f*

_{p}.

## 3.3 Combination of linear and nonlinear depth control methods

*z*=

*z*

_{o}, we can predict the position of the reconstructed image from the equivalent planar pickup and display devices with OPLE lenses. The pickup OPLE lens produces an image of the object at

*z*=

*z*

_{i}where

*z*

_{i}is given in Eq. (3). From this image, elemental images with increased disparity are obtained and then they are digitally zoomed-in. Then, the planar display lenslet array produces an intermediate reconstructed image at

*z*=-

*rz*

_{i}where

*r*is given in Eq. (2). Because of the display OPLE lens, from the Gauss lens law the final reconstructed image is obtained at

*z*=-

*z*

_{r}where

*z*

_{o}varies from ∞ to 0,

*z*

_{r}changes from

*rR*

_{p}

*R*

_{d}/(

*rR*

_{p}+

*R*

_{d}) to 0.

## 4. Other system factors that influence 3-D image depth and size

### 4.1 The use of a modified pickup system

*f*/#. The use of such a camera lens and the planar pickup lenslet array produces the effect of a negatively curved pickup lenslet array, because disparity of elemental images increases. We have to take this effect into account, considering the modified pickup system as a curved pickup system with a curved lenslet array whose radius of curvature is -

*R*

_{c}.

*R*

_{c}equals approximately the distance between the planar pickup lenslet array and the camera lens.

*R*

_{p}as depicted in Fig. 4(b), the actual radius of curvature of the pickup lenslet array is considered to be

*R*

_{p}with

## 4.2 Diverging projection of elemental images

*θ*(e.g., in the azimuthal direction) may not be negligible. In this case, the effect of negatively curved display devices naturally exists even if planar display devices are used, as depicted in Fig. 5(a). Suppose that the horizontal size of overall projected elemental images on the screen is

*S*. Then, one can consider the planar display devices as curved display devices with a radius of curvature -

*R*

_{s}≈-

*S*/

*θ*if the aperture size of the relay optics is much smaller than

*S*. In fact,

*R*

_{s}is approximately equal to the distance between the planar projection screen and the relay optics.

*R*

_{d}as depicted in Fig. 5(b) or in a negatively curved micro-convex-mirror array as in Fig. 5(c). The actual radius of curvature of the display screen in the non-diverging system is:

## 5. Experiments

### 5.1 System description

*R*

_{p}) and 7 cm, respectively. The planar pickup lenslet array we used is made from acrylic, and has 53×53 plano-convex lenslets. Each lenslet element is square-shaped and has a uniform base size of 1.09 mm×1.09 mm, with less than 7.6 µm separating the lenslet elements. The focal length of the lenslets is approximately 3 mm (=

*f*

_{p}). A total of 48×36 elemental images are used in the experiments.

*R*

_{c}≈20 cm. From Eq. (6),

*R*

_{c}=20 cm, when the OPLE lens is not used; and

*α*’s are used:

*α*

_{1}=1,

*α*

_{2}=1.5,

*α*

_{3}=2, and

*α*

_{4}=2.5. A planar micro-convex-mirror array for the projection screen was obtained by coating the convex surface of a lenslet array that is identical to the pickup lenslet array. Light intensity reflectance of the screen is more than 90 %. The focal length of each micro-convex mirror is 0.75 mm (=

*f*

_{d}) in magnitude. Because

*f*

_{p}=3 mm, linear depth squeezing rates are

*r*

_{1}=1/4,

*r*

_{2}=1/6,

*r*

_{3}=1/8, and

*r*

_{4}=1/10 from Eq. (2) for

*α*

_{1},

*α*

_{2}, …,

*α*

_{4}, respectively.

*θ*is approximately 6 degrees in the azimuthal direction. The effect of curved display devices slightly exists. The distance between the screen and the relay optics is approximately 48 cm. Because

*S*=52.3 mm,

*R*

_{s}≈50 mm. From Eq. (7),

*R*

_{d}=∞ in the experiments.

## 5.2 Experimental results

*α*=2.5, digitally zoomed-in elemental images for those in Fig. 8(a) and 8(b) are illustrated in Fig. 8(c) and 8(d), respectively. One can see that the OPLE lens increases disparity between neighboring elemental images.

*r*decreases, reconstructed 3-D images squeeze further in the longitudinal direction and thus disparity between left and right views reduces. The lateral size of reconstructed 3-D images is independent of

*r*. Reconstructed 3-D images at deeper positions are more blurred because the depth-of-focus of the PII system is limited, which is estimated to be 5 cm approximately.

## 6. Discussion and conclusion

## Acknowledgments

## References and links

1. | S. A. Benton, ed., |

2. | T. Okoshi, “Three-dimensional display,” Proc. IEEE |

3. | A. R. L. Travis, “The display of Three-dimensional video images,” Proc. of IEEE |

4. | D. H. McMahon and H. J. Caulfield, “A technique for producing wide-angle holographic displays,” Appl. Opt. |

5. | I. Yamaguchi and T. Zhang, “Phase-shifting digital holography,” Opt. Lett. |

6. | P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “Image reconstruction using electrooptic holography,” |

7. | G. Lippmann, “La photographie integrale,” Comptes-Rendus Academie des Sciences |

8. | H. E. Ives, “Optical properties of a Lippmann lenticulated sheet,” J. Opt. Soc. Am. |

9. | C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. |

10. | N. Davies, M. McCormick, and M. Brewin, “Design and analysis of an image transfer system using microlens arrays,” Opt. Eng. |

11. | F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. |

12. | H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A |

13. | J.-S. Jang, F. Jin, and B. Javidi, “Three-dimensional integral imaging with large depth of focus using real and virtual image fields,” Opt. Lett. |

14. | J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging with nonstationary micro-optics,” Opt. Lett. |

15. | S.-W. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. |

16. | J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. |

17. | Y. Kim, J. Park, H. Choi, S. Jung, S. Min, and B. Lee, “Viewing-angle-enhanced integral imaging system using a curved lens array,” Opt. Express |

18. | J.-S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging using lenslets with non-uniform focal lengths and aperture sizes,” Opt. Lett. |

19. | J.-S. Jang, Y.-S. Oh, and B. Javidi, “Spatiotemporally multiplexed integral imaging projector for large-scale high- resolution three-dimensional display,” Opt. Express |

20. | J.-S. Jang and B. Javidi, “Three-dimensional projection integral imaging using micro-convex-mirror arrays,” Opt. Express |

21. | J. S. Jang and B. Javidi, “Very-large scale integral imaging (VLSII) for 3D display,” to appear in the Journal of Optical Engineering, (2005). |

**OCIS Codes**

(080.0080) Geometric optics : Geometric optics

(100.6890) Image processing : Three-dimensional image processing

(110.6880) Imaging systems : Three-dimensional image acquisition

**ToC Category:**

Research Papers

**History**

Original Manuscript: June 25, 2004

Revised Manuscript: July 23, 2004

Published: August 9, 2004

**Citation**

Ju-Seog Jang and Bahram Javidi, "Depth and lateral size control of three-dimensional images in projection integral imaging," Opt. Express **12**, 3778-3790 (2004)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-12-16-3778

Sort: Journal | Reset

### References

- S. A. Benton, ed., Selected Papers on Three-Dimensional Displays (SPIE Optical Engineering Press, Bellingham, WA, 2001).
- T. Okoshi, ???Three-dimensional display,??? Proc. IEEE 68, 548-564 (1980).
- A.R.L. Travis, ???The display of Three-dimensional video images,??? Proc. of IEEE 85, 1817-1832 (1997).
- D.H. McMahon and H.J. Caulfield, ???A technique for producing wide-angle holographic displays,??? Appl. Opt. 9, 91-96 (1970).
- I. Yamaguchi and T. Zhang, ???Phase-shifting digital holography,??? Opt. Lett. 22, 1268-1270 (1997).
- P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, ???Image reconstruction using electrooptic holography,??? Proceedings of the 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 2003, vol. 1 (IEEE, Piscataway, NJ, 2003) pp. 172-173.
- G. Lippmann, ???La photographie integrale,??? Comptes-Rendus Academie des Sciences 146, 446-451 (1908).
- H. E. Ives, ???Optical properties of a Lippmann lenticulated sheet,??? J. Opt. Soc. Am. 21, 171-176 (1931).
- C.B. Burckhardt, ???Optimum parameters and resolution limitation of integral photography,??? J. Opt. Soc. Am. 58, 71-76 (1968).
- N. Davies, M. McCormick, and M. Brewin, ???Design and analysis of an image transfer system using microlens arrays,??? Opt. Eng. 33, 3624-3633 (1994). [CrossRef]
- F. Okano, H. Hoshino, J. Arai, and I. Yuyama, ???Real-time pickup method for a three-dimensional image based on integral photography,??? Appl. Opt. 36, 1598-1603 (1997).
- H. Hoshino, F. Okano, H. Isono, and I. Yuyama, ???Analysis of resolution limitation of integral photography,??? J. Opt. Soc. Am. A 15, 2059-2065 (1998).
- J.-S. Jang, F. Jin, and B. Javidi, ???Three-dimensional integral imaging with large depth of focus using real and virtual image fields,??? Opt. Lett. 28, 1421-1423 (2003).
- J.-S. Jang and B. Javidi, ???Improved viewing resolution of three-dimensional integral imaging with nonstationary micro-optics,??? Opt. Lett. 27, 324-326 (2002).
- S.-W. Min, B. Javidi, and B. Lee, ???Enhanced three-dimensional integral imaging system by use of double display devices,??? Appl. Opt. 42, 4186-4195 (2003).
- J. Arai, F. Okano, H. Hoshino, and I. Yuyama, ???Gradient-index lens-array method based on real-time integral photography for three-dimensional images,??? Appl. Opt. 37, 2034-2045 (1998).
- Y. Kim, J. Park, H. Choi, S. Jung, S. Min, and B. Lee, ???Viewing-angle-enhanced integral imaging system using a curved lens array,??? Opt. Express 12, 421-429 (2004), <a href="http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-3-421">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-3-421</a>. [CrossRef]
- J.-S. Jang and B. Javidi, ???Large depth-of-focus time-multiplexed three-dimensional integral imaging using lenslets with non-uniform focal lengths and aperture sizes,??? Opt. Lett. 28, 1924-1926 (2003).
- J.-S. Jang, Y.-S. Oh, and B. Javidi, ???Spatiotemporally multiplexed integral imaging projector for large-scale high- resolution three-dimensional display,??? Opt. Express 12, 557-563 (2004), <a href="http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-4-557">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-4-557</a>. [CrossRef]
- J.S. Jang and B. Javidi, ???Very-large scale integral imaging (VLSII) for 3D display,??? to appear in the Journal of Optical Engineering, (2005).
- J.-S. Jang and B. Javidi, ???Three-dimensional projection integral imaging using micro-convex-mirror arrays,??? Opt. Express 12, 1077-1083 (2004), <a href="http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-6-1077">http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-6-1077</a>. [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.