## Anchoring Graph Cuts Towards Accurate Depth Estimation in Integral Images

Journal of Display Technology, Vol. 8, Issue 7, pp. 405-417 (2012)

Acrobat PDF (1502 KB)

### Abstract

Integral imaging is a three-dimensional (3D) imaging technique that allows the displaying of full color images with continuous parallax. Its commercial potential has been increased, due to its ability of presenting to the viewers smooth 3D images, with full parallax, in a wide viewing zone. Being able to extract the inherent 3D information from the planar integral images and produce 3D reconstructions, offers advantages in various applications of immersive entertainment and communications. On this scope, this paper addresses the problem of accurate depth estimation in integral images. The proposed method, relying on the assumption that a pixel is the projection of a 3D imaging ray, aims to specify the first intersection of each pixel's projection ray with the 3D scene in order to assign to it the corresponding depth value. This task is formulated as an energy optimization problem and the graph cuts approach is utilized to solve it. The energy term is twofold; its first part aims to restrict the desired solution to be close to the observed data, i.e., the integral image, while the second one enforces regional smoothness in the depth estimation. This combination offers an accurate and spatially smooth scene structure. The novelty of the paper lies on the framework's formulation as one single optimization procedure and on the way that this optimization is constrained by a set of reliably estimated 3D surface points, called the “anchor points”. Anchoring the optimization results in enhanced depth estimation accuracy, while decreasing the optimization processing burden. The proposed algorithm is evaluated in both synthetic and real integral images consisting of complicated object scenes. A comparison against other state-of-the-art algorithms proves the superiority of the proposed method in terms of depth estimation accuracy.

© 2012 IEEE

**Citation**

Dimitrios Zarpalas, Eleni Fotiadou, Iordanis Biperis, and Petros Daras, "Anchoring Graph Cuts Towards Accurate Depth Estimation in Integral Images," J. Display Technol. **8**, 405-417 (2012)

http://www.opticsinfobase.org/jdt/abstract.cfm?URI=jdt-8-7-405

Sort: Year | Journal | Reset

### References

- A. Aggoun, High-Quality Visual Experience (SpringerLink, 2010) pp. 411-428.
- M. Bertero, P. Boccacci, Introduction to Inverse Problems in Imaging (Institute of Physics Publ., 1998).
- R. Bhotika, D. Fleet, K. Kutulakos, "A probabilistic theory of occupancy and emptiness," ECCV (2002) pp. 112-132.
- B. G. Blundell, A. J. Schwarz, D. K. Horrell, "Volumetric three-dimensional display systems: Their past, present and future," Eng. Sci. Edu. J. 2, 196-200 (1993).
- Y. Boykov, O. Veksler, R. Zabih, "Markov random fields with efficient approximations," IEEE Conf. on Comput. Vision and Pattern Recogn. (1998) pp. 648-655.
- Y. Boykov, O. Veksler, R. Zabih, "Fast approximate energy minimization via graph cuts," IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222-1239 (2001).
- A. Castro, Y. Frauel, B. Javidi, "Integral imaging with large depth of field using an asymmetric phase mask," Opt. Express 15, 10266-10273 (2007).
- S. Cirstea, A. Aggoun, M. McCormick, "Depth extraction from 3D-integral images approached as an inverse problem," Proc. IEEE Int. Symp. on Ind. Electron. (2008) pp. 798-802.
- N. A. Dodgson, EID: Electron. Inf. Displays (, 1997).
- S. M. Faris, "Novel 3D stereoscopic imaging technology," Proc. SPIE (1994) pp. 180.
- Y. Furukawa, J. Ponce, UIUC“High-fidelity image-based modeling,” Tech. Rep. 2006-02 (2006).
- P. Gargallo, P. Sturm, "Bayesian 3D modeling from images using multiple depth maps," CVPR (2005) pp. 885-891.
- C. Hernandez, F. Schmitt, "Silhouette and stereo fusion for 3D object modeling," CVIU 96, 367-392 (2004).
- W. Ijsselsteijn, H. de Ridder, J. Vliegen, "Effects of stereoscopic filming parameters and display duration on the subjective assessment of eye strain," Proc. SPIE Stereosc. Displays and Virtual Reality Syst. VII (2000) pp. 12-22.
- J. Isidoro, S. Sclaroff, "Stochastic refinement of the visual hull to satisfy photometric and silhouette consistency constraints," ICCV (2003) pp. 1335-1342.
- J.-S. Jang, B. Javidi, "Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics," Opt. Lett. 27, 324-326 (2002).
- Y. Kim, J.-H. Park, S.-W. Min, S. Jung, H. Choi, B. Lee, "Wide-viewing-angle integral threedimensional imaging system by curving a screen and a lens array," Appl. Opt. 44, 546-552 (2005).
- V. Kolmogorov, R. Zabih, "Computing visual correspondence with occlusions using graph cuts," ICCV (2001) pp. 508-515.
- V. Kolmogorov, R. Zabih, "What energy functions can be minimized via graph cuts?," Eur. Conf. on Computer Vision (2002).
- V. Kolmogorov, R. Zabih, "Multi-camera scene reconstruction via graph cuts," Eur. Conf. on Computer Vision III, 82-96 (2002).
- K. Kutulakos, S. Seitz, "A theory of shape by space carving," IJCV 38, 199-218 (2000).
- L. Landweber, "An iteration formula for Fredholm integral equations of the first kind," Amer. J. Math. 73, 615-624 (1951).
- M. G. Lippmann, "La photographie integrale," Comptes-Rendus Acad. Sci. 146, 446-551 (1908).
- D. Lowe, "Distinctive image features from scale-invariant keypoints," IJCV 60, 91-110 (2004).
- S. Manolache, S.-Y. Kung, M. McCormick, A. Aggoun, "3D-object space reconstruction from planar recorded data of 3D-integral images," J. VLSI Signal Process. 35, 5-18 (2003).
- S. Manolache, M. McCormick, S. Y. Kung, "Hierarchical adaptive regularization method for depth extraction from planar recording of 3D-integral images," Proc. ICASSP (2001) pp. 1433-1436.
- M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, G. Saavedra, "Integral imaging with improved depth of field by use of amplitude modulated microlens array," Appl. Opt. 43, 5806-5813 (2004).
- M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, G. Saavedra, "Formation of real, orthoscopic integral images by smart pixel mapping," Opt. Express 13, 9175-9180 (2005).
- R. Martínez-Cuenca, H. Navarro, G. Saavedra, B. Javidi, M. Martinez-Corral, "Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system," Opt. Express 15, 16255-16260 (2007).
- R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, B. Javidi, "Enhanced depth of field integral imaging with sensor resolution constraints," Opt. Express 12, 5237-5242 (2004).
- R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, B. Javidi, "Progress in 3D multiperspective display by integral imaging," Proc. IEEE 97, 1067-1077 (2009).
- T. Okoshi, Three Dimensional Imaging Techniques (Academic, 1976).
- J.-H. Park, S. Jung, H. Choi, Y. Kim, B. Lee, "Depth extraction by use of a rectangular lens array and one-dimensional elemental image modification," Appl. Opt. 43, 4882-4895 (2004).
- J.-H. Park, S. Jung, H. Choi, B. Lee, "Viewing-angle-enhanced integral imaging by elemental image resizing and elemental lens switching," Appl. Opt. 41, 6875-6883 (2002).
- S. Roy, I. Cox, "A maximum-flow formulation of the ${\rm N}$ camera stereo correspondence problem," ICCV (1998) pp. 492-499.
- G. Saavedra, R. Martinez-Cuenca, M. Martinez-Corral, H. Navarro, M. Daneshpanah, B. Javidi, "Digital slicing of 3D scenes by Fourier filtering of integral images," Opt. Express 16, 17154-17160 (2008).
- S. Savarese, H. Rushmeier, F. Bernardini, P. Perona, "Shadow carving," ICCV (2001) pp. 190-197.
- D. Scharstein, R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," Int. J. Computer Vision 47, 7-42 (2002).
- S. Seitz, B. Curless, J. Diebel, D. Scharstein, R. Szeliski, "A comparison and evaluation of multi-view stereo reconstruction algorithms," Proc. IEEE Conf. on Computer Vision and Pattern Recogn. (2006) pp. 519-526.
- S. Seitz, C. Dyer, "Photorealistic scene reconstruction by voxel coloring," IJCV 35, 151-173 (1999).
- E. Shechtman, M. Irani, "Recognition of hand gestures using range images," IEEE Conf. on Computer Vision and Pattern Recognition (2007) pp. 1-8.
- E. S. Sinha, M. Pollefeys, "Multi-view reconstruction using photo-consistency and exact silhouette constraints: A maximum-flow formulation," ICCV (2005) pp. 349-356.
- C. Slinger, C. Cameron, M. Stanley, "Computer-generated holography as a generic display technology," Computer 38, 46-53 (2005).
- R. Szeliski, "A multi-view approach to motion and stereo," CVPR (1999) pp. 157-163.
- R. B. A. Tanjung, X. Xu, X. Liang, S. Solanki, Y. Pan, F. Farbiz, B. Xu, T.-C. Chong, "Digital holographic three-dimensional display of 50-Mpixel holograms using a two-axis scanning mirror device," Opt. Eng. 49, 025801 (2010).
- A. N. Tikhonov, V. Y. Arsenin, Solutions of Ill-Posed Problems (Wiley, 1977).
- A. Treuille, A. Hertzmann, S. Seitz, "Example-based stereo with general BRDFs," ECCV (2004) pp. 457-469.
- G. Vogiatzis, C. H. Esteban, P. H. S. Torr, R. Cipolla, "Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency," IEEE Trans. Pattern Anal. Mach.. Intell. 29, 2241-2246 (2007).
- C. Wu, A. Aggoun, M. McCormick, S. Y. Kung, "Depth extraction from unidirectional integral image using a modified multi-baseline technique," Proc. SPIE (2002) pp. 135-143.
- C. H. Wu, M. McCormick, A. Aggoun, S.-Y. Kung, "Depth mapping of integral images through viewpoint image extraction with a hybrid disparity analysis algorithm," J. Display Technol. 4, 101-108 (2008).
- S. Yano, S. Ide, T. Mitsuhashi, H. Thwaites, "A study of visual fatigue and visual comfort for 3-D HDTV/HDTV images," Displays 23, 191-201 (2002).
- J. Y. Chang, H. Park, I. K. Park, K. M. Lee, S. U. Lee, "GPU-friendly multi-view stereo reconstruction using surfel representation and graph cuts," Comput. Vis. Image Understanding 115, 620-634 (2011).
- T. Yu, N. Xu, N. Ahuja, "Shape and view independent reflectance map from multiple views," ECCV (2004) pp. 602-616.
- D. Zarpalas, I. Biperis, E. Fotiadou, E. Lyka, P. Daras, M. G. Strintzis, "Depth estimation in integral images by anchoring optimization techniques," IEEE Int. Conf. on Multimedia & Expo (ICME) (2011).
- G. Zeng, S. Paris, L. Quan, F. Sillion, "Progressive surface reconstruction from images using a local prior," ICCV (2005) pp. 1230-1237.
- Y. Zhang, Q. Ji, W. Zhang, "Multi-view autostereoscopic 3D display," Int. Conf. on Opt. Photon. Energy Eng. (OPEE) (2010).
- L. Zhang, S. Seitz, "Image-based multiresolution shape recovery by surface deformation," SPIE: Videometrics and Optical Methods for 3D Shape Meas. (2001) pp. 51-61.
- C. Zitnick, S.-B. Kang, M. Uyttendaele, S. Winder, R. Szeliski, "High-quality video view interpolation using a layered representation," ACM Trans. Graphics 23, 600-608 (2004).
- B. Micallef, C. J. Debono, R. Farrugia, "Exploiting depth information for efficient multi-view video coding," IEEE Int. Conf. on Multimedia & Expo (ICME) (2011).
- A. Aggoun, "3D Holoscopic video content capture, manipulation and display technologies," 9th Euro-Amer. Workshop on Inf. Opt. (WIO) (2010).

## Cited By |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.