Linear stratified approach using full geometric constraints for 3D scene reconstruction and camera calibration |
Optics Express, Vol. 21, Issue 4, pp. 4456-4474 (2013)
http://dx.doi.org/10.1364/OE.21.004456
Acrobat PDF (5072 KB)
Abstract
This paper presents a new linear framework to obtain 3D scene reconstruction and camera calibration simultaneously from uncalibrated images using scene geometry. Our strategy uses the constraints of parallelism, coplanarity, colinearity, and orthogonality. These constraints can be obtained in general man-made scenes frequently. This approach can give more stable results with fewer images and allow us to gain the results with only linear operations. In this paper, it is shown that all the geometric constraints used in the previous works performed independently up to now can be implemented easily in the proposed linear method. The study on the situations that cannot be dealt with by the previous approaches is also presented and it is shown that the proposed method being able to handle the cases is more flexible in use. The proposed method uses a stratified approach, in which affine reconstruction is performed first and then metric reconstruction. In this procedure, the additional constraints newly extracted in this paper have an important role for affine reconstruction in practical situations.
© 2013 OSA
1. Introduction
1. R. Tsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf tv cameras and lenses,” IEEE Trans. Robot. Autom. 3, 323–344 (1987). [CrossRef]
3. J.-H. Kim and B.-K. Koo, “Convenient calibration method for unsynchronized camera networks using an inaccurate small reference object,” Opt. Express 20, 25292–25310 (2012). [CrossRef] [PubMed]
4. M. Pollefeys and L. V. Gool, “Stratified self-calibration with the modulus constraint,” IEEE Trans. Pattern Anal. Mach. Intell. 21, 707–724 (1999). [CrossRef]
5. M. Pollefeys, L. V. Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vision 59, 207–232 (2004). [CrossRef]
6. T. Moons, L. V. Gool, M. Proesmans, and E. Pauwels, “Affine reconstruction from perspective image pairs with a relative object-camera translation in between,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 77–83 (1996). [CrossRef]
8. L. Agapito, E. Hayman, and I. Reid, “Self-calibration of rotating and zooming cameras,” Int. J. Comput. Vision 45, 107–127 (2001). [CrossRef]
19. K.-Y. K. Wong, G. Zhang, and Z. Chen, “A stratified approach for camera calibration using spheres,” IEEE Trans. Image Process. 20, 305–316 (2011). [CrossRef]
2. Related works
12. C. Rother and S. Carlsson, “Linear multi view reconstruction and camera recovery using a reference plane,” Int. J. Comput. Vision 49, 117–141 (2002). [CrossRef]
14. E. Grossmann and J. Santos-Victor, “Least-squares 3D reconstruction from one or more views and geometric clues,” Comput. Vis. Image Und. 99, 151–174 (2005). [CrossRef]
17. N. Jiang, P. Tan, and L.-F. Cheong, “Symmetric architecture modeling with a single image,” ACM T. Graphic. (Proc. SIGGRAPH Asia) 28 (2009). [CrossRef]
18. F. Mai, Y. S. Hung, and G. Chesi, “Projective reconstruction of ellipses from multiple images,” Pattern Recogn. 43, 545–556 (2010). [CrossRef]
19. K.-Y. K. Wong, G. Zhang, and Z. Chen, “A stratified approach for camera calibration using spheres,” IEEE Trans. Image Process. 20, 305–316 (2011). [CrossRef]
17. N. Jiang, P. Tan, and L.-F. Cheong, “Symmetric architecture modeling with a single image,” ACM T. Graphic. (Proc. SIGGRAPH Asia) 28 (2009). [CrossRef]
3. The infinity homography from parallelism and coplanarity
3.2. Image rectification for a novel framework
3.3. An additional constraint from an actual parallelogram
10. D. Liebowitz and A. Zisserman, “Combining scene and auto-calibration constraints,” in “Proc. IEEE International Conference on Computer Vision,” (Kerkyra, Greece, 1999), pp. 293–300. [CrossRef]
21. Q.-T. Luong and T. Viéville, “Canonical representations for the geometries of multiple perspective views,” Comput. Vis. Image Und. 64, 193–229 (1996). [CrossRef]
3.4. Comparison with related works
13. M. Wilczkowiak, P. Sturm, and E. Boyer, “Using geometric constraints through parallelepipeds for calibration and 3D modelling,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 194–207 (2005). [CrossRef] [PubMed]
13. M. Wilczkowiak, P. Sturm, and E. Boyer, “Using geometric constraints through parallelepipeds for calibration and 3D modelling,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 194–207 (2005). [CrossRef] [PubMed]
4. Reconstruction up to affine transformation
4.1. Parameter reduction using affine invariance
4.2. Comments on linear reconstruction formula
12. C. Rother and S. Carlsson, “Linear multi view reconstruction and camera recovery using a reference plane,” Int. J. Comput. Vision 49, 117–141 (2002). [CrossRef]
12. C. Rother and S. Carlsson, “Linear multi view reconstruction and camera recovery using a reference plane,” Int. J. Comput. Vision 49, 117–141 (2002). [CrossRef]
11. D. Jelinek and C. J. Taylor, “Reconstruction of linearly parameterized models from single images with a camera of unknown focal length,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 767–773 (2001). [CrossRef]
14. E. Grossmann and J. Santos-Victor, “Least-squares 3D reconstruction from one or more views and geometric clues,” Comput. Vis. Image Und. 99, 151–174 (2005). [CrossRef]
14. E. Grossmann and J. Santos-Victor, “Least-squares 3D reconstruction from one or more views and geometric clues,” Comput. Vis. Image Und. 99, 151–174 (2005). [CrossRef]
14. E. Grossmann and J. Santos-Victor, “Least-squares 3D reconstruction from one or more views and geometric clues,” Comput. Vis. Image Und. 99, 151–174 (2005). [CrossRef]
5. Upgrade to metric reconstruction
5.1. Constraints from scene geometry
5.2. Constraints from partial knowledge of camera parameters
5.3. Comparison with related works
13. M. Wilczkowiak, P. Sturm, and E. Boyer, “Using geometric constraints through parallelepipeds for calibration and 3D modelling,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 194–207 (2005). [CrossRef] [PubMed]
6. Outline of the algorithm
7. Experimental results
7.1. Results on synthetic data
7.1.1. Performance evaluation
23. B. K. P. Horn, H. M. Hilden, and S. Negahdaripour, “Closed form solution of absolute orientation using orthonormal matrices,” J. Opt. Soc. Am 5, 1127–1135 (1988). [CrossRef]
7.1.2. Effect of additional constraints
- Corr: The method using vanishing point correspondences.
- NAC: The method using the proposed framework not including the additional constraints.
- AC: The proposed method including the additional constraints.
- F: Using the infinite homography obtained through the fundamental matrix estimation and the projective reconstruction of the cameras and the plane at infinity [20].
- Zhang: The classical method of Zhang [2].
2. Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000). [CrossRef]
7.2. Results on real images
7.2.1. Tower scene
7.2.2. Plant scene
13. M. Wilczkowiak, P. Sturm, and E. Boyer, “Using geometric constraints through parallelepipeds for calibration and 3D modelling,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 194–207 (2005). [CrossRef] [PubMed]
7.2.3. Scene of the Bank of China
13. M. Wilczkowiak, P. Sturm, and E. Boyer, “Using geometric constraints through parallelepipeds for calibration and 3D modelling,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 194–207 (2005). [CrossRef] [PubMed]
7.2.4. Scene of the Casa da Música
13. M. Wilczkowiak, P. Sturm, and E. Boyer, “Using geometric constraints through parallelepipeds for calibration and 3D modelling,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 194–207 (2005). [CrossRef] [PubMed]
8. Conclusions
Acknowledgments
References and links
1. | R. Tsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf tv cameras and lenses,” IEEE Trans. Robot. Autom. 3, 323–344 (1987). [CrossRef] |
2. | Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000). [CrossRef] |
3. | J.-H. Kim and B.-K. Koo, “Convenient calibration method for unsynchronized camera networks using an inaccurate small reference object,” Opt. Express 20, 25292–25310 (2012). [CrossRef] [PubMed] |
4. | M. Pollefeys and L. V. Gool, “Stratified self-calibration with the modulus constraint,” IEEE Trans. Pattern Anal. Mach. Intell. 21, 707–724 (1999). [CrossRef] |
5. | M. Pollefeys, L. V. Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vision 59, 207–232 (2004). [CrossRef] |
6. | T. Moons, L. V. Gool, M. Proesmans, and E. Pauwels, “Affine reconstruction from perspective image pairs with a relative object-camera translation in between,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 77–83 (1996). [CrossRef] |
7. | P. Hammarstedt, F. Kahl, and A. Heyden, “Affine reconstruction from translational motion under various auto-calibration constraints,” J. Math. Imaging Vis. 24, 245–257 (2006). [CrossRef] |
8. | L. Agapito, E. Hayman, and I. Reid, “Self-calibration of rotating and zooming cameras,” Int. J. Comput. Vision 45, 107–127 (2001). [CrossRef] |
9. | R. Cipolla, T. Drummond, and D. P. Robertson, “Camera calibration from vanishing points in images of architectural scenes,” in “Proc. British Machine Vision Conferece,” (Nottingham, England, 1999), pp. 382–391. |
10. | D. Liebowitz and A. Zisserman, “Combining scene and auto-calibration constraints,” in “Proc. IEEE International Conference on Computer Vision,” (Kerkyra, Greece, 1999), pp. 293–300. [CrossRef] |
11. | D. Jelinek and C. J. Taylor, “Reconstruction of linearly parameterized models from single images with a camera of unknown focal length,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 767–773 (2001). [CrossRef] |
12. | C. Rother and S. Carlsson, “Linear multi view reconstruction and camera recovery using a reference plane,” Int. J. Comput. Vision 49, 117–141 (2002). [CrossRef] |
13. | M. Wilczkowiak, P. Sturm, and E. Boyer, “Using geometric constraints through parallelepipeds for calibration and 3D modelling,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 194–207 (2005). [CrossRef] [PubMed] |
14. | E. Grossmann and J. Santos-Victor, “Least-squares 3D reconstruction from one or more views and geometric clues,” Comput. Vis. Image Und. 99, 151–174 (2005). [CrossRef] |
15. | F. C. Wu, F. Q. Duan, and Z. Y. Hu, “An affine invariant of parallelograms and its application to camera calibration and 3D reconstruction,” in “Proc. European Conference on Computer Vision,” (2006), pp. 191–204. |
16. | L. G. de la Fraga and O. Schutze, “Direct calibration by fitting of cuboids to a single image using differential evolution,” Int. J. Comput. Vision 80, 119–127 (2009). [CrossRef] |
17. | N. Jiang, P. Tan, and L.-F. Cheong, “Symmetric architecture modeling with a single image,” ACM T. Graphic. (Proc. SIGGRAPH Asia) 28 (2009). [CrossRef] |
18. | F. Mai, Y. S. Hung, and G. Chesi, “Projective reconstruction of ellipses from multiple images,” Pattern Recogn. 43, 545–556 (2010). [CrossRef] |
19. | K.-Y. K. Wong, G. Zhang, and Z. Chen, “A stratified approach for camera calibration using spheres,” IEEE Trans. Image Process. 20, 305–316 (2011). [CrossRef] |
20. | R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Second Edition (Cambridge University Press, 2003). |
21. | Q.-T. Luong and T. Viéville, “Canonical representations for the geometries of multiple perspective views,” Comput. Vis. Image Und. 64, 193–229 (1996). [CrossRef] |
22. | R. Hartley, “In defence of the 8-point algorithm,” in “Proc. International Conference on Computer Vision,” (Sendai, Japan, 1995), pp. 1064–1070. |
23. | B. K. P. Horn, H. M. Hilden, and S. Negahdaripour, “Closed form solution of absolute orientation using orthonormal matrices,” J. Opt. Soc. Am 5, 1127–1135 (1988). [CrossRef] |
24. | D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice Hall, 2003). |
OCIS Codes
(150.0155) Machine vision : Machine vision optics
(150.1135) Machine vision : Algorithms
(150.1488) Machine vision : Calibration
ToC Category:
Machine Vision
History
Original Manuscript: November 30, 2012
Revised Manuscript: January 28, 2013
Manuscript Accepted: January 29, 2013
Published: February 13, 2013
Citation
Jae-Hean Kim and Bon-Ki Koo, "Linear stratified approach using full geometric constraints for 3D scene reconstruction and camera calibration," Opt. Express 21, 4456-4474 (2013)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-21-4-4456
Sort: Year | Journal | Reset
References
- R. Tsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf tv cameras and lenses,” IEEE Trans. Robot. Autom.3, 323–344 (1987). [CrossRef]
- Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell.22, 1330–1334 (2000). [CrossRef]
- J.-H. Kim and B.-K. Koo, “Convenient calibration method for unsynchronized camera networks using an inaccurate small reference object,” Opt. Express20, 25292–25310 (2012). [CrossRef] [PubMed]
- M. Pollefeys and L. V. Gool, “Stratified self-calibration with the modulus constraint,” IEEE Trans. Pattern Anal. Mach. Intell.21, 707–724 (1999). [CrossRef]
- M. Pollefeys, L. V. Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual modeling with a hand-held camera,” Int. J. Comput. Vision59, 207–232 (2004). [CrossRef]
- T. Moons, L. V. Gool, M. Proesmans, and E. Pauwels, “Affine reconstruction from perspective image pairs with a relative object-camera translation in between,” IEEE Trans. Pattern Anal. Mach. Intell.18, 77–83 (1996). [CrossRef]
- P. Hammarstedt, F. Kahl, and A. Heyden, “Affine reconstruction from translational motion under various auto-calibration constraints,” J. Math. Imaging Vis.24, 245–257 (2006). [CrossRef]
- L. Agapito, E. Hayman, and I. Reid, “Self-calibration of rotating and zooming cameras,” Int. J. Comput. Vision45, 107–127 (2001). [CrossRef]
- R. Cipolla, T. Drummond, and D. P. Robertson, “Camera calibration from vanishing points in images of architectural scenes,” in “Proc. British Machine Vision Conferece,” (Nottingham, England, 1999), pp. 382–391.
- D. Liebowitz and A. Zisserman, “Combining scene and auto-calibration constraints,” in “Proc. IEEE International Conference on Computer Vision,” (Kerkyra, Greece, 1999), pp. 293–300. [CrossRef]
- D. Jelinek and C. J. Taylor, “Reconstruction of linearly parameterized models from single images with a camera of unknown focal length,” IEEE Trans. Pattern Anal. Mach. Intell.23, 767–773 (2001). [CrossRef]
- C. Rother and S. Carlsson, “Linear multi view reconstruction and camera recovery using a reference plane,” Int. J. Comput. Vision49, 117–141 (2002). [CrossRef]
- M. Wilczkowiak, P. Sturm, and E. Boyer, “Using geometric constraints through parallelepipeds for calibration and 3D modelling,” IEEE Trans. Pattern Anal. Mach. Intell.27, 194–207 (2005). [CrossRef] [PubMed]
- E. Grossmann and J. Santos-Victor, “Least-squares 3D reconstruction from one or more views and geometric clues,” Comput. Vis. Image Und.99, 151–174 (2005). [CrossRef]
- F. C. Wu, F. Q. Duan, and Z. Y. Hu, “An affine invariant of parallelograms and its application to camera calibration and 3D reconstruction,” in “Proc. European Conference on Computer Vision,” (2006), pp. 191–204.
- L. G. de la Fraga and O. Schutze, “Direct calibration by fitting of cuboids to a single image using differential evolution,” Int. J. Comput. Vision80, 119–127 (2009). [CrossRef]
- N. Jiang, P. Tan, and L.-F. Cheong, “Symmetric architecture modeling with a single image,” ACM T. Graphic. (Proc. SIGGRAPH Asia) 28 (2009). [CrossRef]
- F. Mai, Y. S. Hung, and G. Chesi, “Projective reconstruction of ellipses from multiple images,” Pattern Recogn.43, 545–556 (2010). [CrossRef]
- K.-Y. K. Wong, G. Zhang, and Z. Chen, “A stratified approach for camera calibration using spheres,” IEEE Trans. Image Process.20, 305–316 (2011). [CrossRef]
- R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Second Edition (Cambridge University Press, 2003).
- Q.-T. Luong and T. Viéville, “Canonical representations for the geometries of multiple perspective views,” Comput. Vis. Image Und.64, 193–229 (1996). [CrossRef]
- R. Hartley, “In defence of the 8-point algorithm,” in “Proc. International Conference on Computer Vision,” (Sendai, Japan, 1995), pp. 1064–1070.
- B. K. P. Horn, H. M. Hilden, and S. Negahdaripour, “Closed form solution of absolute orientation using orthonormal matrices,” J. Opt. Soc. Am5, 1127–1135 (1988). [CrossRef]
- D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice Hall, 2003).
Cited By |
Alert me when this paper is cited |
OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.
Figures
Fig. 1 | Fig. 2 | Fig. 3 |
Fig. 4 | Fig. 5 | Fig. 6 |
Fig. 7 | Fig. 8 | Fig. 9 |
Fig. 10 | Fig. 11 | Fig. 12 |
Fig. 13 | Fig. 14 | |
« Previous Article | Next Article »
OSA is a member of CrossRef.