## Digital slicing of 3D scenes by Fourier filtering of integral images

Optics Express, Vol. 16, Issue 22, pp. 17154-17160 (2008)

http://dx.doi.org/10.1364/OE.16.017154

Acrobat PDF (231 KB)

### Abstract

We present a novel technique to extract depth information from 3D scenes recorded using an Integral Imaging system. The technique exploits the periodic structure of the recorded integral image to implement a Fourier-domain filtering algorithm. A proper projection of the filtered integral image permits reconstruction of different planes that constitute the 3D scene. The main feature of our method is that the Fourier-domain filtering allows the reduction of out-of-focus information, providing the InI system with real optical sectioning capacity.

© 2008 Optical Society of America

## 1. Introduction

4. T. Okoshi, “Three-dimensional displays,” Proc. IEEE **68**, 548–564 (1980). [CrossRef]

5. F. Okano, H. Hoshino, J. Arai, and I. Yayuma, “Real time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. **36**, 1598–1603 (1997). [CrossRef] [PubMed]

9. S. Jung, J.-H. Park, H. Choi, and B. Lee, “Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement,” Opt. Express **12**, 1346–1356 (2003). [CrossRef]

10. J. Arai, M. Okui, T. Yamashita, and F. Okano, “Integral three-dimensional television using a 2000-scanning-line video system,” Appl. Opt. **45**, 1704–1712 (2006). [CrossRef] [PubMed]

11. J.-S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperture sizes,” Opt. Lett. **28**, 1924–1926 (2003). [CrossRef] [PubMed]

13. R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, “Extended depth-of-field 3-D display and visualization by combination of amplitude-modulated microlenses and deconvolution tools,” J. Disp. Technol. **1**, 321–327 (2005). [CrossRef]

14. J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, “Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging,” Opt. Lett. **29**, 2734–2736 (2004). [CrossRef] [PubMed]

15. H. Choi, S.-W. Min, S. Yung, J.-H. Park, and B. Lee, “Multiple viewing zone integral image using dynamic barrier array fro three-dimensional displays,” Opt. Express **11**, 927–932 (2003). [CrossRef] [PubMed]

16. R. Martínez-Cuenca, H. Navarro, G. Saavedra, B. Javidi, and M. Martínez-Corral, “Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system,” Opt. Express **15**, 16255–16260 (2007). [CrossRef] [PubMed]

17. J.-S Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. **27**, 324–326 (2002). [CrossRef]

18. M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Multifacet structure of observed reconstructed integral images,” J. Opt. Soc. Am. A **22**, 597–603 (2005). [CrossRef]

19. J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, “Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging,” Opt. Lett. **29**, 2734–2736 (2004). [CrossRef] [PubMed]

20. W. Matusik and H. Pfister. “3D TV: A Scalable System for Real-Time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes.” ACM Trans. Graph. **23**, 814–824 (2004). [CrossRef]

21. H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, “Real-time 3D image-guided navigation system based on integral videography,” Proc. SPIE **4615**, 36–44 (2002). [CrossRef]

22. Y. Frauel and B. Javidi, “Digital Three-Dimensional Image Correlation by Use of Computer-Reconstructed Integral Imaging,” Appl. Opt. **41**, 5488–5496 (2002). [CrossRef] [PubMed]

25. C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, “Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm,” J. Electron. Imaging **14**, 023018 (2005). [CrossRef]

26. J.-S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. **27**, 1144–1146 (2002). [CrossRef]

28. B. Javidi, R. Ponce-Díaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. **31**, 1106–1108 (2006). [CrossRef] [PubMed]

## 2. Fourier filtering of integral images

_{s}. Assuming that paraxial approximation holds, is clear that each microlens provides a scaled image of the object, and therefore the integral image is composed by a set of equally spaced replicas of the object. In Fig. 1 we have drawn a scheme of the pickup. For the sake of simplicity, the scheme and also the following equations have been described in one dimension. The extension to 2D is straightforward.

*N*, and that the central microlens is aligned with the optical axis of the pickup system. We label this lens as L

_{x}_{0}and the other microlenses as L

_{m},

*m*being the integer lens index ranging from -(

*N*-1)/2 to (

_{x}*N*-1)/2. As shown in Fig 1, the integral image of a point object placed at (

_{x}*x*,

_{S}*z*) is composed of a series of replicas positioned at:

_{S}*M*=-

_{S}*g*/

*z*is the scale factor between the object and the image plane. The pickup period,

_{S}*T*, is the distance between replicas of

_{S}*S*, and depends on the MLA pitch,

*p*, through

30. R. Martínez-Cuenca, A. Pons, G. Saavedra, M. Martínez-Corral, and B. Javidi, “Optically-corrected elemental images for undistorted integral image display,” Opt. Express **14**, 9657–9663 (2006). [CrossRef] [PubMed]

_{m}(z

_{S})-

*mp*|<

*p*/2, are able to record the replica of S. This constraint limits the maximum,

*m*, and the minimum,

^{max}*m*, index of lenses that record the image of S. The number microlenses that are contributing to the integral image is then

^{min}*n*(

_{x}*z*)=

_{S}*m*-

^{max}*m*+1. Thus, the integral image of a plane object centered at

^{min}*S*can be calculated, as

*x*=(

*n*-1)

_{x}*T*, and

_{S}*O*(

*x*) is the object intensity distribution. The Fourier transform of the expression above is

*z*can be written as

_{R}*sinc*functions in the Fourier domain. Since this function does not fall sharply to zero, the signals generated by objects at different depths cannot be perfectly discriminated.

## 3. Volumetric reconstruction of Fourier filtered integral image

24. S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express **12**, 483–491 (2004). [CrossRef] [PubMed]

_{1}lies within the field of view of 7 elemental images, whereas the intensity information about R

_{2}is only carried by 6 rays. This difference must be taken into account in the averaging process. Note, besides, that ray cones emanated from a single object point at the reconstruction plane would recombine again by the back projection method to accurately recreate the intensity of the object point. However, rays emanated from the object points away from the reconstruction plane would mix with their neighbors resulting in a defocused effect. Thus, with computational reconstruction one is able to get a focus image of an object at the correct reconstruction distance. The rest of the scene appears blurred.

*k*-th elemental image be denoted by

*O*

_{k}(

*x*). For image reconstruction, each filtered elemental image is flipped and shifted according to the reconstruction distance and superimposed to generate the desired plane. Therefore the final reconstruction plane,

*I*(

*x,z*) consists of partial overlap of flipped and shifted filtered elemental images as:

_{R}*K*denote the number of elemental images acquired. Besides,

*R*compensates for intensity variation due to different distances from the object plane to elemental image

*O*

_{k}(

*x*) on the sensor and is given by:

*z*)

_{s}+g^{2}and can be assumed to be constant for a particular reconstruction distance.

## 4. Experimental results

*f*=3

*mm*, pitch

*p*=1.03

*mm*and gap

*g*=3.10

*mm*. The cars labeled with 6 and 2 were located approximately 70mm and 90mm away from the MLA.

*z*=70 mm and

_{R}*z*=92 mm. Each filtered integral image is then used as an input for the subsequent volumetric reconstruction. This allows slice by slice reconstruction of the 3D scene showing optical sectioning effects.

_{R}## 5. Conclusions.

## Acknowledgment

## References and Links

1. | M. G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. (Paris) |

2. | H. E. Ives, “Optical properties of a Lippman lenticulated sheet,” J. Opt. Soc. Am. |

3. | C. B. Burckhardt, “Optimum parameters and resolution limitation of Integral Photography,” J. Opt. Soc. Am. |

4. | T. Okoshi, “Three-dimensional displays,” Proc. IEEE |

5. | F. Okano, H. Hoshino, J. Arai, and I. Yayuma, “Real time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. |

6. | J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. |

7. | H. Arimoto and B. Javidi, “Integral 3D imaging with digital reconstruction,” Opt. Lett. |

8. | J.-S Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. |

9. | S. Jung, J.-H. Park, H. Choi, and B. Lee, “Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement,” Opt. Express |

10. | J. Arai, M. Okui, T. Yamashita, and F. Okano, “Integral three-dimensional television using a 2000-scanning-line video system,” Appl. Opt. |

11. | J.-S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperture sizes,” Opt. Lett. |

12. | R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, “Enhanced depth of field integral imaging with sensor resolution constraints,” Opt. Express |

13. | R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, “Extended depth-of-field 3-D display and visualization by combination of amplitude-modulated microlenses and deconvolution tools,” J. Disp. Technol. |

14. | J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, “Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging,” Opt. Lett. |

15. | H. Choi, S.-W. Min, S. Yung, J.-H. Park, and B. Lee, “Multiple viewing zone integral image using dynamic barrier array fro three-dimensional displays,” Opt. Express |

16. | R. Martínez-Cuenca, H. Navarro, G. Saavedra, B. Javidi, and M. Martínez-Corral, “Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system,” Opt. Express |

17. | J.-S Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. |

18. | M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Multifacet structure of observed reconstructed integral images,” J. Opt. Soc. Am. A |

19. | J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, “Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging,” Opt. Lett. |

20. | W. Matusik and H. Pfister. “3D TV: A Scalable System for Real-Time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes.” ACM Trans. Graph. |

21. | H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, “Real-time 3D image-guided navigation system based on integral videography,” Proc. SPIE |

22. | Y. Frauel and B. Javidi, “Digital Three-Dimensional Image Correlation by Use of Computer-Reconstructed Integral Imaging,” Appl. Opt. |

23. | S. Yeom and B. Javidi, “Three-dimensional distortion-tolerant object recognition using integral imaging,” Opt. Express |

24. | S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express |

25. | C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, “Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm,” J. Electron. Imaging |

26. | J.-S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. |

27. | S.-H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express |

28. | B. Javidi, R. Ponce-Díaz, and S.-H. Hong, “Three-dimensional recognition of occluded objects by using computational integral imaging,” Opt. Lett. |

29. | T. Wilson, ed. |

30. | R. Martínez-Cuenca, A. Pons, G. Saavedra, M. Martínez-Corral, and B. Javidi, “Optically-corrected elemental images for undistorted integral image display,” Opt. Express |

**OCIS Codes**

(100.6890) Image processing : Three-dimensional image processing

(110.4190) Imaging systems : Multiple imaging

(110.6880) Imaging systems : Three-dimensional image acquisition

(150.5670) Machine vision : Range finding

**ToC Category:**

Image Processing

**History**

Original Manuscript: July 8, 2008

Revised Manuscript: September 11, 2008

Manuscript Accepted: September 11, 2008

Published: October 13, 2008

**Citation**

G. Saavedra, R. Martinez-Cuenca, M. Martinez-Corral, H. Navarro, M. Daneshpanah, and B. Javidi, "Digital slicing of 3D scenes by Fourier filtering of integral images," Opt. Express **16**, 17154-17160 (2008)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-16-22-17154

Sort: Year | Journal | Reset

### References

- M. G. Lippmann, "Epreuves reversibles donnant la sensation du relief," J. Phys. (Paris) 7, 821-825 (1908).
- H. E. Ives, "Optical properties of a Lippman lenticulated sheet," J. Opt. Soc. Am. 21, 171-176 (1931). [CrossRef]
- C. B. Burckhardt, "Optimum parameters and resolution limitation of Integral Photography," J. Opt. Soc. Am. 58, 71-76 (1968). [CrossRef]
- T. Okoshi, "Three-dimensional displays," Proc. IEEE 68, 548-564 (1980). [CrossRef]
- F. Okano, H. Hoshino, J. Arai, and I. Yayuma, "Real time pickup method for a three-dimensional image based on integral photography," Appl. Opt. 36, 1598-1603 (1997). [CrossRef] [PubMed]
- J. Arai, F. Okano, H. Hoshino, and I. Yuyama, "Gradient-index lens-array method based on real-time integral photography for three-dimensional images," Appl. Opt. 37,2034-2045 (1998). [CrossRef]
- H. Arimoto and B. Javidi, "Integral 3D imaging with digital reconstruction," Opt. Lett. 26, 157-159 (2001). [CrossRef]
- J.-S Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002). [CrossRef]
- S. Jung, J.-H. Park, H. Choi and B. Lee, "Viewing-angle-enhanced integral three-dimensional imaging along all directions without mechanical movement," Opt. Express 12, 1346-1356 (2003). [CrossRef]
- J. Arai, M. Okui, T. Yamashita, and F. Okano, "Integral three-dimensional television using a 2000-scanning-line video system," Appl. Opt. 45, 1704-1712 (2006) [CrossRef] [PubMed]
- J.-S. Jang and B. Javidi, "Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperture sizes," Opt. Lett. 28, 1924-1926 (2003). [CrossRef] [PubMed]
- R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Enhanced depth of field integral imaging with sensor resolution constraints," Opt. Express 12, 5237-5242 (2004). [CrossRef] [PubMed]
- R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, "Extended depth-of-field 3-D display and visualization by combination of amplitude-modulated microlenses and deconvolution tools," J. Disp. Technol. 1, 321-327 (2005). [CrossRef]
- J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, "Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging," Opt. Lett. 29, 2734-2736 (2004). [CrossRef] [PubMed]
- H. Choi, S.-W. Min, S. Yung, J.-H. Park, and B. Lee, "Multiple viewing zone integral image using dynamic barrier array fro three-dimensional displays," Opt. Express 11, 927-932 (2003). [CrossRef] [PubMed]
- R. Martínez-Cuenca, H. Navarro, G. Saavedra, B. Javidi, and M. Martínez-Corral, "Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system," Opt. Express 15, 16255-16260 (2007). [CrossRef] [PubMed]
- J.-S Jang and B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002). [CrossRef]
- M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, "Multifacet structure of observed reconstructed integral images," J. Opt. Soc. Am. A 22, 597-603 (2005). [CrossRef]
- J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, "Depth-enhanced three-dimensional two-dimensional convertible display based on modified integral imaging," Opt. Lett. 29, 2734-2736 (2004). [CrossRef] [PubMed]
- W. Matusik and H. Pfister, "3D TV: A Scalable System for Real-Time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes," ACM Trans. Graph. 23, 814-824 (2004). [CrossRef]
- H. Liao, S. Nakajima, M. Iwahara, N. Hata, and S. I. y T. Dohi, "Real-time 3D image-guided navigation system based on integral videography," Proc. SPIE 4615, 36-44 (2002). [CrossRef]
- Y. Frauel and B. Javidi, "Digital Three-Dimensional Image Correlation by use of Computer-Reconstructed Integral Imaging," Appl. Opt. 41, 5488-5496 (2002) [CrossRef] [PubMed]
- S. Yeom and B. Javidi, "Three-dimensional distortion-tolerant object recognition using integral imaging," Opt. Express 12, 5795-5809 (2004) [CrossRef] [PubMed]
- S.-H. Hong, J.-S. Jang, and B. Javidi, "Three-dimensional volumetric object reconstruction using computational integral imaging," Opt. Express 12, 483-491 (2004). [CrossRef] [PubMed]
- C. Wu, A. Aggoun, M. McCormick, and S. Y. Kung, "Depth measurement from integral images through viewpoint image extraction and a modified multibaseline disparity analysis algorithm," J. Electron. Imaging 14, 023018 (2005) [CrossRef]
- J.-S. Jang and B. Javidi, "Three-dimensional synthetic aperture integral imaging," Opt. Lett. 27, 1144-1146 (2002). [CrossRef]
- S.-H. Hong and B. Javidi, "Distortion-tolerant 3D recognition of occluded objects using computational integral imaging," Opt. Express 14, 12085- 12095 (2006). [CrossRef] [PubMed]
- B. Javidi, R. Ponce-Díaz, and S.-H. Hong, "Three-dimensional recognition of occluded objects by using computational integral imaging," Opt. Lett. 31, 1106-1108 (2006). [CrossRef] [PubMed]
- T. Wilson, ed., Confocal Microscopy (Academic, London 1990).
- R. Martínez-Cuenca, A. Pons, G. Saavedra, M. Martínez-Corral and B. Javidi, "Optically-corrected elemental images for undistorted integral image display," Opt. Express 14, 9657-9663 (2006). [CrossRef] [PubMed]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

### Multimedia

Multimedia Files | Recommended Software |

» Media 1: AVI (4846 KB) | QuickTime |

« Previous Article | Next Article »

OSA is a member of CrossRef.