OSA's Digital Library

Applied Optics

Applied Optics

APPLICATIONS-CENTERED RESEARCH IN OPTICS

  • Editor: Joseph N. Mait
  • Vol. 52, Iss. 4 — Feb. 1, 2013
  • pp: 546–560

Advances in three-dimensional integral imaging: sensing, display, and applications [Invited]

Xiao Xiao, Bahram Javidi, Manuel Martinez-Corral, and Adrian Stern  »View Author Affiliations


Applied Optics, Vol. 52, Issue 4, pp. 546-560 (2013)
http://dx.doi.org/10.1364/AO.52.000546


View Full Text Article

Enhanced HTML    Acrobat PDF (1833 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

Three-dimensional (3D) sensing and imaging technologies have been extensively researched for many applications in the fields of entertainment, medicine, robotics, manufacturing, industrial inspection, security, surveillance, and defense due to their diverse and significant benefits. Integral imaging is a passive multiperspective imaging technique, which records multiple two-dimensional images of a scene from different perspectives. Unlike holography, it can capture a scene such as outdoor events with incoherent or ambient light. Integral imaging can display a true 3D color image with full parallax and continuous viewing angles by incoherent light; thus it does not suffer from speckle degradation. Because of its unique properties, integral imaging has been revived over the past decade or so as a promising approach for massive 3D commercialization. A series of key articles on this topic have appeared in the OSA journals, including Applied Optics. Thus, it is fitting that this Commemorative Review presents an overview of literature on physical principles and applications of integral imaging. Several data capture configurations, reconstruction, and display methods are overviewed. In addition, applications including 3D underwater imaging, 3D imaging in photon-starved environments, 3D tracking of occluded objects, 3D optical microscopy, and 3D polarimetric imaging are reviewed.

© 2013 Optical Society of America

OCIS Codes
(110.6880) Imaging systems : Three-dimensional image acquisition
(120.2040) Instrumentation, measurement, and metrology : Displays
(150.6910) Machine vision : Three-dimensional sensing

ToC Category:
Imaging Systems

History
Original Manuscript: September 10, 2012
Manuscript Accepted: September 14, 2012
Published: January 24, 2013

Virtual Issues
(2013) Advances in Optics and Photonics
(2014) Advances in Optics and Photonics
Vol. 8, Iss. 3 Virtual Journal for Biomedical Optics

Citation
Xiao Xiao, Bahram Javidi, Manuel Martinez-Corral, and Adrian Stern, "Advances in three-dimensional integral imaging: sensing, display, and applications [Invited]," Appl. Opt. 52, 546-560 (2013)
http://www.opticsinfobase.org/ao/abstract.cfm?URI=ao-52-4-546


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. A. Sokolov, “Autostereoscopy and integral photography by Professor Lippmann’s method,” in Izd. MGU (Moscow State University, 1911).
  2. H. E. Ives, “Optical properties of a Lippman lenticulated sheet,” J. Opt. Soc. Am. 21, 171 (1931). [CrossRef]
  3. C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. A 58, 71–74 (1968). [CrossRef]
  4. T. Okoshi, “Three-dimensional displays,” Proc. IEEE 68, 548–564 (1980). [CrossRef]
  5. T. Okoshi, Three-Dimensional Imaging Techniques (Academic, 1976).
  6. B. Javidi, F. Okano, and J. Y. Son, Three-Dimensional Imaging, Visualization, and Display (Springer, 2009).
  7. L. Yang, M. McCormick, and N. Davies, “Discussion of the optics of a new 3-D imaging system,” Appl. Opt. 27, 4529–4534 (1988). [CrossRef]
  8. F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE 94, 490–501 (2006). [CrossRef]
  9. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. 37, 2034–2045 (1998). [CrossRef]
  10. H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A 15, 2059–2065 (1998). [CrossRef]
  11. T. Mishina, “3D television system based on integral photography,” in Proceedings of the Picture Coding Symposium (PCS), 2010 (IEEE, 2010), p. 20.
  12. J. Arai, F. Okano, M. Kawakita, M. Okui, Y. Haino, M. Yoshimura, M. Furuya, and M. Sato, “Integral three-dimensional television using a 33-megapixel imaging system,” J. Disp. Technol. 6, 422–430 (2010). [CrossRef]
  13. O. Matoba, E. Tajahuerce, and B. Javidi, “Real-time three-dimensional object recognition with multiple perspectives imaging,” Appl. Opt. 40, 3318–3325 (2001). [CrossRef]
  14. S. Kishk and B. Javidi, “Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,” Opt. Express 11, 3528–3541 (2003). [CrossRef]
  15. S. H. Hong and B. Javidi, “Distortion-tolerant 3D recognition of occluded objects using computational integral imaging,” Opt. Express 14, 12085–12095 (2006). [CrossRef]
  16. R. Schulein, C. M. Do, and B. Javidi, “Distortion-tolerant 3D recognition of underwater objects using neural networks,” J. Opt. Soc. Am. A 27, 461–468 (2010). [CrossRef]
  17. M. DaneshPanah and B. Javidi, “Profilometry and optical slicing by passive three-dimensional imaging,” Opt. Lett. 34, 1105–1107 (2009). [CrossRef]
  18. J. H. Park and K. M. Jeong, “Frequency domain depth filtering of integral imaging,” Opt. Express 19, 18729–18741 (2011). [CrossRef]
  19. A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE 94, 591–607 (2006). [CrossRef]
  20. S. Yeom, B. Javidi, and E. Watson, “Three-dimensional distortion-tolerant object recognition using photon-counting integral imaging,” Opt. Express 15, 1513–1533 (2007). [CrossRef]
  21. B. Tavakoli, B. Javidi, and E. Watson, “Three dimensional visualization by photon counting computational integral imaging,” Opt. Express 16, 4426–4436 (2008). [CrossRef]
  22. I. Moon and B. Javidi, “Three-dimensional recognition of photon-starved events using computational integral imaging and statistical sampling,” Opt. Lett. 34, 731–733 (2009). [CrossRef]
  23. M. DaneshPanah, B. Javidi, and E. A. Watson, “Three dimensional object recognition with photon counting imagery in the presence of noise,” Opt. Express 18, 26450–26460 (2010). [CrossRef]
  24. D. Aloni, A. Stern, and B. Javidi, “Three-dimensional photon counting integral imaging reconstruction using penalized maximum likelihood expectation maximization,” Opt. Express 19, 19681–19687 (2011). [CrossRef]
  25. S. H. Hong and B. Javidi, “Three-dimensional visualization of partially occluded objects using integral imaging,” J. Disp. Technol. 1, 354–359 (2005). [CrossRef]
  26. I. Moon and B. Javidi, “Three-dimensional visualization of objects in scattering medium by use of computational integral imaging,” Opt. Express 16, 13080–13089 (2008). [CrossRef]
  27. M. Cho and B. Javidi, “Three-dimensional visualization of objects in turbid water using integral imaging,” J. Disp. Technol. 6, 544–547 (2010). [CrossRef]
  28. B. Javidi, I. Moon, and S. Yeom, “Three-dimensional identification of biological microorganism using integral imaging,” Opt. Express 14, 12096–12108 (2006). [CrossRef]
  29. J. S. Jang and B. Javidi, “Three-dimensional integral imaging of micro-objects,” Opt. Lett. 29, 1230–1232 (2004). [CrossRef]
  30. M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235, 144–162 (2009). [CrossRef]
  31. D. Shin, M. Cho, and B. Javidi, “Three-dimensional optical microscopy using axially distributed image sensing,” Opt. Lett. 35, 3646–3648 (2010). [CrossRef]
  32. Y. Zhao, X. Xiao, M. Cho, and B. Javidi, “Tracking of multiple objects in unknown background using Bayesian estimation in 3D space,” J. Opt. Soc. Am. A 28, 1935–1940 (2011). [CrossRef]
  33. X. Xiao, B. Javidi, G. Saavedra, M. Eismann, and M. Martinez-Corral, “Three-dimensional polarimetric computational integral imaging,” Opt. Express 20, 15481–15488 (2012). [CrossRef]
  34. C. Wheatstone, “Contributions to the physiology of vision.—Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision,” Philos. Trans. R. Soc. Lond. 128, 371–394 (1838). [CrossRef]
  35. W. Rollmann, “Zwei neue stereoskopische Methoden,” Ann. Phys. 166, 186–187 (1853). [CrossRef]
  36. D. S. Kim, S. M. Park, J. H. Jung, and D. C. Hwang, “51.2: new 240 Hz driving method for full HD & high quality 3D LCD TV,” SID Symp. Dig. Tech. Pap. 41, 762–765 (2010). [CrossRef]
  37. S. S. Kim, B. H. You, H. Choi, B. H. Berkeley, D. G. Kim, and N. D. Kim, “World’s first 240 Hz TFT‐LCD technology for full‐HD LCD‐TV and its application to 3D display,” SID Symp. Dig. Tech. Pap. 40, 424–427 (2009). [CrossRef]
  38. H. Kang, S. D. Roh, I. S. Baik, H. J. Jung, W. N. Jeong, J. K. Shin, and I. J. Chung, “3.1: a novel polarizer glasses‐type 3D displays with a patterned retarder,” SID Symp. Dig. Tech. Pap. 41, 1–4 (2010). [CrossRef]
  39. C. Slinger, C. Cameron, and M. Stanley, “Computer-generated holography as a generic display technology,” Computer 38, 46–53 (2005). [CrossRef]
  40. R. B. A. Tanjung, X. Xu, X. Liang, S. Solanki, Y. Pan, F. Farbiz, B. Xu, and T. C. Chong, “Digital holographic three-dimensional display of 50-Mpixel holograms using a two-axis scanning mirror device,” Opt. Eng. 49, 025801(2010). [CrossRef]
  41. P. A. Blanche, A. Bablumian, R. Voorakaranam, C. Christenson, W. Lin, T. Gu, D. Flores, P. Wang, W. Y. Hsieh, and M. Kathaperumal, “Holographic three-dimensional telepresence using large-area photorefractive polymer,” Nature 468, 80–83 (2010). [CrossRef]
  42. M. Holroyd, I. Baran, J. Lawrence, and W. Matusik, “Computing and fabricating multilayer models,” ACM Trans. Graph. 30, 187 (2011). [CrossRef]
  43. A. Marraud and M. Bonnet, “Restitution of stereoscopic picture by means of a lenticular sheet,” Proc. SPIE 0402, 129–132 (1983).
  44. Mashitani, “Autostereoscopic video display with a parallax barrier having oblique apertures,” U.S. patent 7,317,494(8January2008).
  45. H. J. Lee, H. Nam, J. D. Lee, H. W. Jang, M. S. Song, B. S. Kim, J. S. Gu, C. Y. Park, and K. H. Choi, “A high resolution autostereoscopic display employing a time division parallax barrier,” SID Symp. Dig. Tech. Pap. 37, 81–84 (2006). [CrossRef]
  46. G. Hamagishi, “Analysis and improvement of viewing conditions for two‐view and multi‐view displays,” SID Symp. Dig. Tech. Pap. 40, 340–343 (2009). [CrossRef]
  47. T. Inoue and H. Ohzu, “Accommodative responses to stereoscopic three-dimensional display,” Appl. Opt. 36, 4509–4515 (1997). [CrossRef]
  48. F. L. Kooi and A. Toet, “Visual comfort of binocular and 3D displays,” Displays 25, 99–108 (2004). [CrossRef]
  49. G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–825 (1908). [CrossRef]
  50. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997). [CrossRef]
  51. J. Arai, H. Hoshino, M. Okui, and F. Okano, “Effects of focusing on the resolution characteristics of integral photography,” J. Opt. Soc. Am. A 20, 996–1004 (2003). [CrossRef]
  52. D. H. Shin, E. S. Kim, and B. Lee, “Computational reconstruction of three-dimensional objects in integral imaging using lenslet array,” Jpn. J. Appl. Phys. 44, 8016–8018 (2005). [CrossRef]
  53. B. Tavakoli, M. Daneshpanah, B. Javidi, and E. Watson, “Performance of 3D integral imaging with position uncertainty,” Opt. Express 15, 11889–11902 (2007). [CrossRef]
  54. J. H. Park, G. Baasantseren, N. Kim, G. Park, J. M. Kang, and B. Lee, “View image generation in perspective and orthographic projection geometry based on integral imaging,” Opt. Express 16, 8800–8813 (2008). [CrossRef]
  55. J. Y. Son, S. H. Kim, D. S. Kim, B. Javidi, and K. D. Kwack, “Image-forming principle of integral photography,” J. Disp. Technol. 4, 324–331 (2008). [CrossRef]
  56. Y. T. Lim, J. H. Park, K. C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17, 19253–19263 (2009). [CrossRef]
  57. M. U. Erdenebat, G. Baasantseren, and J. H. Park, “Full-parallax 360 degrees integral imaging display,” in Proceedings of the International Meeting on Information Display (Korean Information Display Society, 2010), pp. 812–813.
  58. H. Navarro, R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, “3D integral imaging display by smart pseudoscopic-to-orthoscopic conversion (SPOC),” Opt. Express 18, 25573–25583 (2010). [CrossRef]
  59. H. Geng, Q. H. Wang, L. Li, and D. H. Li, “An integral-imaging three-dimensional display with wide viewing angle,” J. SID 19, 679–684 (2011).
  60. M. Cho, and B. Javidi, “Optimization of 3D integral imaging system parameters,” IEEE J. Disp. Technol. 8, 357–360 (2012). [CrossRef]
  61. A. Yöntem and L. Onural, “Integral imaging using phase-only LCoS spatial light modulators as Fresnel lenslet arrays,” J. Opt. Soc. Am. A 28, 2359–2375 (2011). [CrossRef]
  62. H. Navarro, R. Martínez-Cuenca, A. Molina-Martín, M. Martínez-Corral, G. Saavedra, and B. Javidi, “Method to remedy image degradations due to facet braiding in 3D integral-imaging monitors,” J. Disp. Technol. 6, 404–411 (2010). [CrossRef]
  63. F. Okano, J. Arai, H. Hoshino, and I. Yuyama, “Three-dimensional video system based on integral photography,” Opt. Eng. 38, 1072–1077 (1999). [CrossRef]
  64. N. Davies, M. McCormick, and L. Yang, “Three-dimensional imaging systems: a new development,” Appl. Opt. 27, 4520–4528 (1988). [CrossRef]
  65. E. H. Adelson and J. Y. A. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992). [CrossRef]
  66. M. Levoy, “Light fields and computational imaging,” Computer 39, 46–55 (2006). [CrossRef]
  67. J. H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48, H77–H94 (2009). [CrossRef]
  68. J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. 27, 1144–1146 (2002). [CrossRef]
  69. M. DaneshPanah, B. Javidi, and E. A. Watson, “Three dimensional imaging with randomly distributed sensors,” Opt. Express 16, 6368–6377 (2008). [CrossRef]
  70. R. Schulein, M. DaneshPanah, and B. Javidi, “3D imaging with axially distributed sensing,” Opt. Lett. 34, 2012–2014 (2009). [CrossRef]
  71. X. Xiao, M. DaneshPanah, M. Cho, and B. Javidi, “3D integral imaging using sparse sensors with unknown positions,” J. Disp. Technol. 6, 614–619 (2010). [CrossRef]
  72. Y. Igarashi, H. Murata, and M. Ueda, “3D display system using a computer generated integral photography,” Jpn. J. Appl. Phys. 17, 1683–1684 (1978). [CrossRef]
  73. M. Halle, “Multiple viewpoint rendering,” in Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (1998), pp. 243–254.
  74. R. Yang, X. Huang, S. Li, and C. Jaynes, “Toward the light field display: autostereoscopic rendering via a cluster of projectors,” IEEE Trans. Vis. Comput. Graph. 14, 84–96 (2008).
  75. M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Multifacet structure of observed reconstructed integral images,” J. Opt. Soc. Am. A 22, 597–603 (2005). [CrossRef]
  76. M. Martínez-Corral, H. Navarro, R. Martínez-Cuenca, G. Saavedra, and B. Javidi, “Full parallax 3-D TV with programmable display parameters,” Opt. Photon. News 22(12), 50–50 (2011). [CrossRef]
  77. R. Martínez-Cuenca, H. Navarro, G. Saavedra, B. Javidi, and M. Martinez-Corral, “Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system,” Opt. Express 15, 16255–16260 (2007). [CrossRef]
  78. H. Choi, S. W. Min, S. Jung, J. H. Park, and B. Lee, “Multiple-viewing-zone integral imaging using a dynamic barrier array for three-dimensional displays,” Opt. Express 11, 927–932 (2003). [CrossRef]
  79. M. Miura, J. Arai, T. Mishina, M. Okui, and F. Okano, “Integral imaging system with enlarged horizontal viewing angle,” Proc. SPIE 8384, 83840O (2012). [CrossRef]
  80. S. H. Hong, J. S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12, 483–491 (2004). [CrossRef]
  81. H. Arimoto, and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. 26, 157–159 (2001). [CrossRef]
  82. V. Vaish, M. Levoy, R. Szeliski, C. L. Zitnick, and S. B. Kang, “Reconstructing occluded surfaces using synthetic apertures: stereo, focus and robust measures,” in Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 2331–2338.
  83. S. Yeom, B. Javidi, and E. Watson, “Photon counting passive 3D image sensing for automatic target recognition,” Opt. Express 13, 9310–9330 (2005). [CrossRef]
  84. X. Xiao and B. Javidi, “3D Photon counting integral imaging with unknown sensor positions,” J. Opt. Soc. Am. A 29, 767–771 (2012). [CrossRef]
  85. V. Y. Panin, G. L. Zeng, and G. T. Gullberg, “Total variation regulated EM algorithm,” IEEE Trans. Nucl. Sci. 46, 2202–2210 (1999). [CrossRef]
  86. P. J. Green, “Bayesian reconstructions from emission tomography data using a modified EM algorithm,” IEEE Trans. Med. Imag. 9, 84–93 (1990). [CrossRef]
  87. A. Stern, D. Aloni, and B. Javidi, “Experiments with three-dimensional integral imaging under low light levels,” IEEE Photonics J. 4, 1188–1195 (2012). [CrossRef]
  88. D. Shin, M. Daneshpanah, and B. Javidi, “Generalization of three-dimensional N-ocular imaging systems under fixed resource constraints,” Opt. Lett. 37, 19–21 (2012). [CrossRef]
  89. S. Sinha, D. Steedly, R. Szeliski, M. Agrawala, and M. Pollefeys, “Interactive 3D architectural modeling from unordered photo collections,” ACM Trans. Graph. 27, 1–10 (2008). [CrossRef]
  90. A. Gotchev, G. Akar, T. Capin, D. Strohmeier, and A. Boev, “Three-dimensional media for mobile devices,” Proc. IEEE 99, 708–741 (2011). [CrossRef]
  91. B. Javidi, S. H. Hong, and O. Matoba, “Multidimensional optical sensor and imaging system,” Appl. Opt. 45, 2986–2994 (2006). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited