OSA's Digital Library

Optics Express

Optics Express

  • Editor: Andrew M. Weiner
  • Vol. 22, Iss. 2 — Jan. 27, 2014
  • pp: 1287–1301
« Show journal navigation

Absolute three-dimensional shape measurement using coded fringe patterns without phase unwrapping or projector calibration

William Lohry, Vincent Chen, and Song Zhang  »View Author Affiliations


Optics Express, Vol. 22, Issue 2, pp. 1287-1301 (2014)
http://dx.doi.org/10.1364/OE.22.001287


View Full Text Article

Acrobat PDF (9255 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

This paper presents a novel stereo-phase-based absolute three-dimensional (3D) shape measurement that requires neither phase unwrapping nor projector calibration. This proposed method can be divided into two steps: (1) obtain a coarse disparity map from the quality map; and (2) refine the disparity map using wrapped phase. Fringe patterns are modified to encode the quality map for efficient and accurate stereo matching. Experiments demonstrated that the proposed method could achieve high-quality 3D measurement even with extremely low-quality fringe patterns.

© 2014 Optical Society of America

1. Introduction

Triangulation-based three-dimensional (3D) shape measurement can be classified into two categories: the passive method (e.g. stereo vision) and the active method (e.g., structured light). In a passive stereo system, two images captured from different perspectives are used to detect corresponding points in a scene to obtain 3D geometry [1

1. D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Intl J. Comp. Vis. 47(1–3), 7–42 (2002). [CrossRef]

, 2

2. U. R. Dhond and J. K. Aggarwal, “Structure from stereo-a review,” IEEE Trans. Systems, Man. and Cybernetics 19(6), 1489–1510 (1989). [CrossRef]

]. Detecting corresponding points between two stereo images is a well-studied problem in stereo vision. Since a corresponding point pair must lie on an epipolar line, the captured images are often rectified so that the epipolar lines run across each row [3

3. R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge University Press, ISBN: 0521623049, 2000).

]. This allows a method of finding corresponding points using a “sliding window” approach, which defines the similarity of a match using cost, correlation, or probability. The difference between the horizontal position of the point in the left image and that in the right image is called the disparity. This disparity can be directly converted into 3D geometry.

Standard cost-based matching approaches rely on the texture difference between a source point in one image with a target point in the other [4

4. T. Kanade and M. Okutomi, “A stereo matching algorithm with an adaptive window: Theory and experiment,” IEEE Trans. Patt. Analy. and Mach. Intellig. 16(9), 920–932 (1994). [CrossRef]

]. The cost represents the difference in intensity between the two windows on the epipolar line and is used to weigh various matches. In a winner-takes-all approach, the disparity will be determined from the point in the right image that has the least cost with that of the source point in the left.

In addition to local methods, a number of global and semi-global methods have been suggested [5

5. V. Kolmogorov and R. Zabih, “Multi-camera scene reconstruction via graph cuts,” in Euro Conf. Comp. Vis., pp. 82–96 (2002).

8

8. F. Besse, C. Rother, A. W. Fitzgibbon, and J. Kautz, “PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation,” Intl J. Comp. Vis. pp. 1–12 (Springer2013).

]. One method that worked especially well was the probabilistic model named Efficient LArge-scale Stereo (ELAS) [9

9. A. Geiger, M. Roser, and R. Urtasun, “Efficient Large-Scale Stereo Matching,” 6492, 25–38 (2011).

]. In this method, a number of support points from both images are chosen based on their response to a 3 × 3 Sobel filter. Groups of points are compared between images, and a Bayesian model determines their likelihood of matching. Since the ELAS method is piecewise continuous, it works particularly well for objects with little texture variation.

Passive stereo methods, despite recent advances, still suffer from the fundamental limitation of the method: finding corresponding pairs between two natural images. This requirement hinders the ability of this method to accurately and densely reconstruct many real-world objects such as uniform white surfaces. An alternative to a dual-camera stereo method is to replace one camera with a projector and actively project a desired texture on the object surface for stereo matching [10

10. J. Salvi, S. Fernandez, T. Pribanic, and X. Llado, “A state of the art in structured light patterns for surface profilometry,” Patt. Recogn. 43(8), 2666–2680 (2010). [CrossRef]

]. This method is typically referred to as structured light. The structured systems uses computer generated patterns that carries codewords through certain codification strategies. The coded patterns can be the pseudo-random dots (or speckle patterns, such as the Microsoft Kinect [11

11. Z. Zhang, “Microsoft Kinect Sensor and Its Effect,” IEEE Multimedia 19(2), 4–10 (2012). [CrossRef]

] uses), the binary structured patterns. If the codewords are unique, the correspondence between the projector and the camera is uniquely determined, and 3D information can be calculated through triangulation once the system is calibrated. One of the major benefits of using the pseudo-random codification approach is that it is easy to understand and easy to implement for 3D shape measurement. The major disadvantage to this technique, however, is that it is difficult for such techniques to achieve high-spatial resolution as it is limited by the projector’s resolution in both the u and v directions.

The structured light systems often use binary patterns for 3D shape measurement with some achieving real-time capability [12

12. O. Hall-Holt and S. Rusinkiewicz, “Stripe Boundary Codes for Real-Time Structured-Light Range Scanning of Moving Objects,” in The 8th IEEE International Conference on Computer Vision, pp. II: 359–366 (2001).

, 13

13. S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D Model Acquisition,” ACM Trans. Graph. 21(3), 438–446 (2002). [CrossRef]

]. Because these techniques only uses 0s and 1s for codification, they have the following advantages: (1) the coding and decoding algorithms are very simple; (2) processing can be performed quickly since the algorithm is simple; and (3) it is very robust to the noise since only two levels are used. Comparing with the codification with the pseudo-random dots, it could achieve higher spatial resolution. This is because such a technique uses vertical or horizontal stripes for codification, its spatial resolution is only limited by the projector’s resolution in either u or v direction, but not both. However, the stripe width must be larger than one projector’s pixel, and to achieve projector pixel spatial resolution, a lot binary patterns (typically a lot more than three patterns are required).Therefore, it is very difficult for these methods to reach pixel level spatial resolution at very high speeds.

The binary coded patterns are continuous in one direction (along the stripe) but discrete in the other, limiting its achievable spatial resolution. Higher spatial resolution could be achieved by using continuous patterns in both directions, and the patterns are typically sinusoidal in nature. The method of using sinusoidal structured patterns for codification is also called digital fringe projection or DFP. Sinusoidal patterns are often analyzed using phase-shifting algorithms. In optical metrology, the phase-shifting-based structured-light method is extensively used due to its accuracy and speed [14

14. D. Malacara, ed., Optical Shop Testing, 3 (John Wiley and Sons, New York, NY, 2007). [CrossRef]

]. For a DFP system, instead of finding corresponding point using intensity of the structured patterns, it uses phase as a constraint to solve for (x, y, z) coordinates pixel by pixel if the system is calibrated [15

15. S. Zhang, “Recent progresses on real-time 3-D shape measurement using digital fringe projection techniques,” Opt. Laser Eng. 48(2), 149–158 (2010). [CrossRef]

]. Because of their overwhelmingly advantageous features over the other types of structured light techniques, the DFP techniques have been extensively adopted in numerous disciplines [16

16. G. Geng, “Structured-light 3D surface imaging: a tutorial,” Advances in Opt. and Photonics 3(2), 128–160 (2011). [CrossRef]

,17

17. S. Gorthi and P. Rastogi, “Fringe projection techniques: Whither we are?” Opt. Laser. Eng. 48, 133–140 (2010). [CrossRef]

]. The major advantages are: (1) High spatial resolution. The phase-shifting technique permits pixel-by-pixel 3D shape measurement, making it possible to achieve camera pixel-level spatial resolution; (2) Resistance to ambient light influences. Instead of directly utilizing intensity, the phase-shifting method analyzes phase information of the structured pattern. The ambient light influence is automatically canceled out, albeit the signal-to-noise ratio (SNR) might be sacrificed if the ambient light is too strong comparing with the projection light; (3) Lower sensitivity to surface reflectivity variations. Typically, the phase-shifting method computes the phase using the arctangent function point-by-point, and the influence of surface reflectivity information is also canceled out because it is constant for each pixel; (4) High-speed 3D shape measurement. Since the whole measurement area can be captured and processed once, this technique, as well as other structured light techniques, can achieve high measurement speed; and (5) Can achieve high measurement accuracy. Unlike other structured light techniques, the phase-shifting based method allows precise sub-pixel correspondence between the projector and the camera without any interpolation. Therefore, theoretically, it could achieve highly accurate 3D shape measurement if calibration is properly performed.

In the meantime, comparing with other structured light methods, the DFP method suffers from several problems. Firstly, the phase obtained from the phase-shifting algorithm usually ranges from 0 to 2π with 2π discontinuities. To recover 3D shape, the continuous phase must be obtained, usually requiring spatial or temporal phase unwrapping. The spatial phase unwrapping, by looking at neighboring pixel phase information, detects the 2π jumps and removes them by adding or subtracting multiples of 2π [18

18. D. C. Ghiglia and M. D. Pritt, eds., Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (John Wiley and Sons, New York, 1998).

]. Even though numerous spatial phase unwrapping algorithms have been developed, a spatial phase unwrapping algorithm only works for “smooth” surfaces without abrupt changes (inducing more than π between two neighboring pixels) from one point to the other. Therefore, the spatial phase unwrapping cannot be used for large step-height or isolated object measurement. Furthermore, the phase obtained by spatial phase unwrapping only provides relative phase that only gives the relative shape of the object to a reference point; and to recover the absolute geometry, absolute phase is required. The absolute phase can be obtained by temporal phase unwrapping. Instead of comparing the phase values of neighboring pixels, a temporal phase unwrapping algorithm uses clues from other phase values in the same camera pixel. Over the past many years, many temporal phase unwrapping methods have been developed including two- [19

19. K. Creath, “Step height measurement using two-wavelength phase-shifting interferometry,” Appl. Opt. 26(14), 2810–2816 (1987). [CrossRef] [PubMed]

] or multi-frequency [20

20. Y.-Y. Cheng and J. C. Wyant, “Multiple-Wavelength Phase Shifting Interferometry,” Appl. Opt. 24, 804–807 (1985). [CrossRef]

] phase-shifting, the gray-coding plus phase-shifting [21

21. G. Sansoni, M. Carocci, and R. Rodella, “Three-Dimensional Vision Based on a Combination of Gray-Code and Phase-Shift Light Projection: Analysis and Compensation of the Systematic Errors,” Appl. Opt. 38, 6565–6573 (1999). [CrossRef]

], composite of random or stair image with phase-shifting [22

22. U. R. Dhond and J. K. Aggarwal, “Real-time scalable depth sensing with hybrid structured light illumination,” IEEE Trans. Imag. Proc. 23(1), 97–109 (2013).

, 23

23. S. Zhang, “Composite phase-shifting algorithm for absolute phase measurement,” Opt. Laser Eng. 50(11), 1538–1541 (2012). [CrossRef]

] methods. However, all these temporal phase unwrapping requires more images to be captured, slowing down the measurement speeds. For high-speed applications, such a method is usually not desirable.

Secondly, the projector has to be accurately calibrated [24

24. X. Chen, J. Xi, Y. Jin, and J. Sun, “Accurate calibration for a camera–projector measurement system based on structured light projection,” Opt. Laser Eng. 47(3), 310–319 (2009). [CrossRef]

]. Even though numerous digital video projector calibration methods [25

25. Q. Hu, P. S. Huang, Q. Fu, and F. P. Chiang, “Calibration of a 3-D Shape Measurement System,” Opt. Eng. 42(2), 487–493 (2003). [CrossRef]

27

27. R. Legarda-Sáenz, T. Bothe, and W. P. Jüptner, “Accurate Procedure for the Calibration of a Structured Light System,” Opt. Eng. 43(2), 464–471 (2004). [CrossRef]

] have been developed, accurate projector calibration remains difficult. This is mainly because, unlike a camera, a projector cannot directly capture images like a camera. We pioneered the method of converting the structured light system calibration to the standard stereo system calibration by enabling the projector to “capture” images like a camera [28

28. S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng. 458), 083,601 (2006). [CrossRef]

]. Despite some recent advancements [29

29. W. Gao, L. Wang, and Z. Hu, “Flexible method for structured light system calibration,” Opt. Eng. 478), 083,602 (2008). [CrossRef]

31

31. Z. Li, Y. Shi, C. Wang, and Y. Wang, “Accurate calibration method for a structured light system,” Opt. Eng. 475), 053,604 (2008). [CrossRef]

] on structured light calibration, we still found great difficulty accurately calibrating structured light systems, especially for high-speed 3D shape measurement systems. Furthermore, all these developed good calibration methods have the following requirements: (1) the projector can be manipulated pixel by pixel; (2) the projector is at least nearly focused when perform any measurement; and (3) the absolute correspondence between camera pixel and the projector pixel (or at least line). However, for a physical grating or slide projector, the first requirement cannot be satisfied; for a system with an out-of-focus projector, the second requirement cannot be satisfied; and for a high-speed 3D shape measurement system, as aforementioned, satisfying requirement 3 usually slows down the measurement. Therefore, calibrating generic structured light system remains challenging.

Lastly, since the traditional DFP-based method recovers 3D geometry directly from the phase, the phase quality is essential to measurement accuracy: any noise or distortion on the phase will be reflected on the final 3D measurement. Unlike the structured light system using a binary coding method, the DFP system requires calibrating the nonlinearity of the projection and compensating for the errors associated with it. Even though nonlinearity correction is relatively mature with many methods developed [32

32. S. Kakunai, T. Sakamoto, and K. Iwata, “Profile measurement taken with liquid-crystal grating,” Appl. Opt. 38(13), 2824–2828 (1999). [CrossRef]

37

37. S. Zhang and S.-T. Yau, “Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector,” Appl. Opt. 46(1), 36–43 (2007). [CrossRef]

], our research found that the nonlinearity of the projector actually changes over time, complicating the problem since a regular nonlinearity calibration is usually necessary [15

15. S. Zhang, “Recent progresses on real-time 3-D shape measurement using digital fringe projection techniques,” Opt. Laser Eng. 48(2), 149–158 (2010). [CrossRef]

].

For standard camera-projector based structured light systems, color-coded structured patterns can be used to speed up the measurement [38

38. J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng. 4512), 013,602 (2006). [CrossRef]

45

45. P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring,” Opt. Eng. 38, 1065–1071 (1999). [CrossRef]

]. However, the object surface color could influence, to a various degree, the measurement quality. This is because the color of the object could change the coded information carried on by the patterns. For example, if the object surface is blue, the information carried on by blue or red colors could be completely (at least partially) lost. Furthermore, the color coupling problem [38

38. J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng. 4512), 013,602 (2006). [CrossRef]

] further complicates the measurement system development. Therefore, the black-and-white coded patterns are typically preferrable for high accuracy measurement, and to achieve high speed, these patterns are usually switched quickly [46

46. S. Zhang and P. S. Huang, “High-resolution real-time three-dimensional shape measurement,” Opt. Eng. 4512), 123,601 (2006). [CrossRef]

].

To mitigate some of problems associated with passive stereo or actively structured light methods, the natural approach is to combine these two methods together: using two cameras and one projector. Over the years, different methods have been developed. In general, they use random dots [11

11. Z. Zhang, “Microsoft Kinect Sensor and Its Effect,” IEEE Multimedia 19(2), 4–10 (2012). [CrossRef]

, 47

47. Y. Huang, Y. Shang, Y. Liu, and H. Bao, Handbook of 3D Machine Vision: Optical Metrology and Imaging, chap. 3D shapes from Speckle, pp. 33–56, 1 (CRC, 2013). [CrossRef]

], bandlimited random patterns [49

49. A. Wiegmann, H. Wagner, and R. Kowarschik, “Human face measurement by projecting bandlimited random patterns,” Opt. Express 14(17), 7692–7698 (2006). [CrossRef] [PubMed]

], binary-coded patterns [48

48. L. Zhang, N. Snavely, B. Curless, and S. M. Seitz, “Spacetime Faces: High-Resolution Capture for Modeling and Animation,” ACM Trans. Graph. 23(3), 548–558 (2004). [CrossRef]

, 50

50. J. Davis, R. Ramamoorthi, and S. Rusinkiewicz, “Spacetime Stereo: A Unifying Framework for Depth from Triangulation,” IEEE Trans. Patt. Anal. and Mach.e Intell. 27(2), 1–7 (2005).

], color structured patterns [51

51. L. Zhang, B. Curless, and S. Seitz, “Spacetime Stereo: Shape Recovery for Dynamic Scenes,” in Proc. Comp. Vis. Patt. Recogn., pp. 367–374 (2003).

, 52

52. W. Jang, C. Je, Y. Seo, and S. W. Lee, “Structured-light stereo: Comparative analysis and integration of structured-light and active stereo for measuring dynamic shape,” Opt. Laser Eng. 51(11), 1255–1264 (2013). [CrossRef]

], or phase-shifted sinusoidal fringe patterns [53

53. X. Han and P. Huang, “Combined stereovision and phase shifting method: use of a visibility-modulated fringe pattern,” in SPIE Europe Optical Metrology, pp. 73,893H (2009).

55

55. K. Liu and Y. Wang, “Phase channel multiplexing pattern strategy for active stereo vision,” in Intl Conf. 3D Imaging (IC3D), pp. 1–8 (2012). [CrossRef]

]. The overviews can be found in the literature [56

56. J. Salvi, J. Pages, and J. Batlle, “Pattern codification strategies in structured light systems,” Patt. Recogn. 37(4), 827–849 (2004). [CrossRef]

58

58. C. Bräuer-Burchardt, M. Möller, C. Munkelt, M. Heinze, P. Kühmstedt, and G. Notni, “On the accuracy of point correspondence methods in three-dimensional measurement systems using fringe projection,” Opt. Eng. 526), 063,601 (2013). [CrossRef]

]. Typically, the random-pattern-based methods could achieve high speed but the achievable spatial resolution is relatively low. The binary-coding method is relatively robust to noise, yet spatial resolution is usually limited by the both the projector and the camera, albeit is higher than that achieved by those random-dots based methods. The use of color could speed the measurement, yet induces the problems associated with color (e.g., color coupling, susceptibility to object surface color) [38

38. J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng. 4512), 013,602 (2006). [CrossRef]

]. The phase-shifting based DFP system can achieve high spatial resolution (only limited by the camera); and could also achieve high speeds since only three patterns are required for dense 3D shape measurement. Therefore, for both high spatial resolution and high speed 3D shape measurements, the DFP technique is usually adopted.

The phase-based method becomes more powerful if neither spatial nor temporal phase unwrapping is necessary. Taking advantage of the geometric constraints of the trio sensors (two cameras and one projector), References [59

59. Z. Li, K. Zhong, Y. Li, X. Zhou, and Y. Shi, “Multiview phase shifting: a full-resolution and high-speed 3D measurement framework for arbitrary shape dynamic objects,” Opt. Lett. 38(9), 1389–1391 (2013). [CrossRef] [PubMed]

61

61. C. Bräuer-Burchardt, P. Kühmstedt, and G. Notni, “Phase unwrapping using geometric constraints for high-speed fringe projection based 3D measurements,” in SPIE Optical Metrology 2013, pp. 878,906 (2013).

] presented 3D shape measurement techniques without phase unwrapping. However, similar to prior phase-based methods, these methods require projector calibration, which is usually not easy and even more difficult for nonlinear projection sources. Furthermore, the geometric constraint usually requires globally backward and forward checking for matching point location, limiting its speed and capability of measuring sharp changing surface geometries.

We propose a method to alleviate the problems associated with the aforementioned techniques. This proposed method combines the advantages of the stereo approach and the phase-based approach: using a stereo matching algorithm to obtain the coarse disparity map to avoid the global searching problem associated with the method in [59

59. Z. Li, K. Zhong, Y. Li, X. Zhou, and Y. Shi, “Multiview phase shifting: a full-resolution and high-speed 3D measurement framework for arbitrary shape dynamic objects,” Opt. Lett. 38(9), 1389–1391 (2013). [CrossRef] [PubMed]

]; and using the local wrapped phase information to further refine the coarse disparity for higher measurement accuracy. Furthermore, the proposed method does not require any geometric constraint imposed by the projector, and thus no projector calibration is required, further simplifying the system development.

Section 2 explains the principle of the proposed method. Section 3 shows the experimental results. Section 4 discusses the advantages and shortcomings of the proposed method, and Sec. 5 summarizes this paper.

2. Principle

2.1. Three-step phase-shifting algorithm

For high-speed applications, a three phase-shifting algorithm is desirable. For a three-step phase-shifting algorithm with equal phase shifts, three fringe patterns can be described as
I1(x,y)=I+Icos(ϕ2π/3),
(1)
I2(x,y)=I+Icos(ϕ),
(2)
I3(x,y)=I+Icos(ϕ+2π/3),
(3)
where I′(x, y) represents the average intensity, I″(x, y) the intensity modulation, and ϕ(x, y) the phase to be solved for. Solving these three equations leads to
ϕ(x,y)=tan1[3(I1I3)2I2I1I3],
(4)
γ(x,y)=II=3(I1I3)2+(2I2I1I3)2I1+I2+I3.
(5)
Here γ(x, y) is the data modulation that represents the quality of each data point with 1 being the best, and its map is referred to as the quality map.

2.2. Combination of statistical random pattern with phase-shifting fringe pattern

The encoded pattern was generated using band-limited 1/f noise where 120pixels<f<15pixels and with intensity Ip(x, y) such that 0.5 < Ip(x, y) < 1. Here Ip(x, y) is randomly generated. In Eqs. (1)(3), I″(x, y) was changed to Ip(x, y)I″(x, y). The modified fringe images are described as
I1(x,y)=I+Ip(x,y)Icos(ϕ2π/3),
(6)
I2(x,y)=I+Ip(x,y)Icos(ϕ),
(7)
I3(x,y)=I+Ip(x,y)Icos(ϕ+2π/3).
(8)

Figure 1 illustrates the encoded pattern Ip(x, y) and one of the modified fringe patterns. Since the encoded pattern is still centered around the same average intensity value, the captured texture image or phase should not be affected in theory, albeit the phase signal to noise ratio may be lower, and the nonlinearity of the projection system may affect texture image quality. Furthermore, any naturally occurring quality map changes caused by object texture or proximity to the projector will be visible from both of the cameras, canceling the effect.

Fig. 1 Example of 1/f noise used for encoded pattern. (a) Encoded pattern, Ip(x, y); (b) modified fringe pattern

The 2D varying pattern can improve the cost distinction between a correct match and the other possible disparities. While random pattern stereo matching algorithms have been proposed [62

62. M. Maruyama and S. Abe, “Range sensing by projecting multiple slits with random cuts,” IEEE Trans. Patt. Analysis Mach. Intellig. 15(6), 647–651 (1993). [CrossRef]

, 63

63. K. Konolige, “Projected texture stereo,” inIEEE Intl Conf. Rob. Auto., pp. 148–155 (2010).

], they have been used for the final disparity calculation rather than as an intermediary to matching phase. In this paper, the random pattern is used to match corresponding phase points between two images, without the need for global phase unwrapping. Once the corresponding points have been determined, refinement of the disparity map can proceed using only the wrapped phase locally.

2.3. Disparity map determination

ELAS [9

9. A. Geiger, M. Roser, and R. Urtasun, “Efficient Large-Scale Stereo Matching,” 6492, 25–38 (2011).

] is used to obtain an initial coarse disparity map. Since the pattern encoded in γ(x, y) provides great distinctness for many of the pixels, it produces a much more accurate map than just the texture I′(x, y). The encoded random pattern can be converted to an 8-bit grayscale image by scaling the intensity values for quality between 0 and 1 for input into ELAS.

The coarse disparity map provides a rough correspondence between images. However, it must still be refined to obtain a sub-pixel disparity. While the refinement could be performed on the random pattern itself, refinement using phase has several advantages: the phase is less sensitive to noise and monotonically increases across the image even in the presence of some level of higher-order harmonics.

Unlike the spatial or temporal unwrapping methods that require absolute phase, the proposed method only requires a local unwrapping window along a 3- to 5-pixel line. In a correct match, both the source and the target will lie within π radians, and this constraint can be used to properly align the phases.

The refinement step is defined as finding the sub-pixel shift τ such that the center of the target phase matches the center of the source phase:
xtarget(ϕ)+τ=xsource(ϕ)
(9)

The relationship between the x-coordinate and the phase should locally have the same underlying curve for both the target and the source except for the displacement τ, so x(ϕ) can be fitted using a polynomial anϕn, where both the target and the source share the same parameters an for n > 0.
xtarget(ϕ)=a0t+a1ϕ+a2ϕ2+a3ϕ3
(10)
xsource(ϕ)=a0s+a1ϕ+a2ϕ2+a3ϕ3
(11)
We used third-order polynomial fittings to refine the disparity. The sub-pixel shift will be the displacement when ϕsource = 0, yielding τ=a0ta0s and a final disparity of d = dcoarseτ, where dcoarse is the coarse disparity for that pixel.

3. Experiments

We developed a hardware system to verify the proposed technique, as shown in Fig. 2. This system includes an LED digital-light-processing (DLP) projector (model: Dell M109S) and two CMOS cameras (model: Point Grey Research Flea3 FL3-U3-13Y3M-C) with 12 mm lenses (model: Computar 08K). The projector resolution is 800 × 600. The cameras use USB 3.0 connection and are set as capturing images with a resolution of 800 × 600. The two cameras were calibrated using the asymmetric circle grid and the open-source software package OpenCV. Throughout the whole experiments, the projector remained uncalibrated (neither nonlinear gamma nor optical system parameters). It is important to note that the baseline distance between these two cameras d is very small (approximately 49.5 mm).

Fig. 2 Photograph of the developed hardware system.

Figures 3 and 4 show the experimental results of measuring a smooth spherical surface (radius of approximately 51 mm) shown in Fig. 4(a), that usually fails the traditional stereo system since there is no substantial texture variation from one point to the other. Figure 3(a) shows one of the three phase-shifted fringe patterns captured from the left camera. From three fringe patterns, the wrapped phase map, shown in Fig. 3(b), and the quality map, shown in Fig. 3(c), can be obtained. Similar images were also obtained for the right camera shown in the second row of Fig. 3. From the quality maps, one may notice that, besides the encoded random information, there are some vertical structured patterns which are usually not present in a normal DFP system. This is caused by the projector’s nonlinear effect as the projector is not calibrated.

Fig. 3 Experimental results of a smooth sphere. (a) One for three fringe patterns from left camera; (b) wrapped phase map; (c) quality map, γ(x, y); (d)–(f) corresponding images for the right camera.
Fig. 4 Experimental results of a smooth sphere. (a) Photograph of measured sphere; (b) coarse disparity map using ELAS algorithm; (c) Refined disparity map using wrapped phase; (d) 3D result using (b); (e) 3D reconstruction using (c); (f) unwrapped phase after removing gross slope.

Figure 4(b) shows the coarse disparity map that was obtained from two camera’s quality maps employing the ELAS algorithm; and Figure 4(c) shows the refined disparity map using the wrapped phase. From the disparity maps, 3D shapes were reconstructed using the calibrated stereo camera system. Figure 4(d) and 4(e) respectively shows the result from the coarse disparity map, and the refined disparity map. These results demonstrate that the 3D reconstruction from coarse disparity map is very rough (i.e., not detailed) even with the encoded quality map. The 3D refined results using wrapped phase gives mored detailed and accurate measurement. It should be noted that since the fringe patterns are encoded and the projector is not calibrated, the phase quality is very poor. Figure 4(f) shows the unwrapped phase map after removing its gross slope, and the phase was unwrapped by using a multiple-frequency temporal phase unwrapping algorithm [64

64. Y. Wang and S. Zhang, “Superfast multifrequency phase-shifting technique with optimal pulse width modulation,” Opt. Express 19(6), 5143–5148 (2011).

].

To further demonstrate the differences among these different approaches for smooth spherical surface measurement, we performed further analysis. Since the projector was not directly calibrated throughout the whole experiments, the 3D result from the conventional reference-plane-based method cannot achieve the same measurement accuracy as the stereo-based method. Our extensive experimental evaluation found that the reference-plane-based method provides better result than the projector-calibration method such as the one we developed [28

28. S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng. 458), 083,601 (2006). [CrossRef]

]. This is probably because the baseline of the camera and the projector [Fig. 2] is very small (approximately 25 mm), making the projector calibration inaccurate and thus reducing 3D shape measurement accuracy. Therefore, even though the reference-plane-based method cannot provide great measurement accuracy, we still chose this method as a comparison.

To compare them fairly, the sphere was normalized to reflect the relative error rather than absolute error for all these results, as illustrated in Fig. 5. Figure 5(a) shows the normalized cross-section of the 3D result shown in Fig. 4(d); Figure 5(b) shows the same normalized cross-section of the 3D result shown in Fig. 4(e); and Figure 5(c) shows the normalized cross-section of the 3D result shown in Fig. 4(f).

Fig. 5 Surface measurement error for different methods. (a) Normalized cross-section of the 3D result shown in Fig. 4(d); (b) Normalized cross-section of the 3D result shown in Fig. 4(e); and (c) Normalized cross-section of the 3D result shown in Fig. 4(f); (d)–(f) shows the difference between the 3D result with the fitted smooth surface.

Since the spherical surface we measured is smooth, we then fit these curves with smooth curve to determine the error between the normalized 3D results and the smoothed ones. Figures 5(d)–5(e) show the results. It should be noted the scale on Fig. 5(f) is 10 times that of Fig. 5(d) and Fig. 5(e). These data clearly demonstrate that even from the poor quality of the phase, the high-quality 3D shape can still be properly reconstructed with the proposed method. This is the because, unlike a single camera DFP system where 3D information is directly extracted from phase, the proposed stereo system only uses the phase as a reference constraint to establish correspondence.

To evaluate their accuracy, the measured results were compared against the ideal sphere. Figure 6 shows the comparing results for the refined 3D shape [Fig. 6(a)] and the 3D shape from the standard reference-plane-based calibration method [Fig. 6(b)]. It clearly shows that the proposed method [Fig. 6(c)] is substantially more accurate than the standard reference plane based method [Fig. 6(d)]: root mean square (rms) error of 0.16 mm vs 2.27 mm.

Fig. 6 Comparing measurement results with ideal spheres. (a) Cross sections of the refined 3D shape and the ideal sphere; (b) Cross sections of the 3D shape from the reference-plane-based calibration method and the ideal sphere; (c) Difference error for (a) (rms error is approximately 0.16 mm); (d) Difference error for (b) (rms error is approximately 2.27 mm).

To verify that the proposed method can measure more complex and absolute shape of the object, Figure 7 shows the results of simultaneously measuring two separate statues of size 62mm × 42 mm × 40 mm. Again, the phase quality is of very poor quality as shown in Fig. 7(b), but high-quality 3D shape can be obtained using the proposed method shown in Fig. 7(d) and Fig. 7(f), which is substantially better than the result [Fig. 7(c) and Fig. 7(e)] obtained from the ELAS stereo matching algorithm. These experimental results further confirmed that the proposed method could be applied to arbitrary 3D shape measurement, even for two separate objects, providing a novel method of measurement 3D absolute shape without the requirements of high-quality phase, projector calibration, or spatial phase unwrapping.

Fig. 7 Experimental results of more complex objects. (a) Photograph of measured statues; (b) quality map showing encoded pattern; (c) coarse disparity map using ELAS algorithm; (d) Refined disparity map using wrapped phase; (e) closeup view of (c); (f) closeup view of (d).

4. Discussions

By embedding the statistical pattern into the phase-shifted fringe patterns for 3D shape recovery, the proposed active-stereo technique has the following major merits:
  • High spatial resolution. The proposed method inherits one of the merits of the phase-shifting-based methods: achieving camera-pixel spatial resolution.
  • Absolute phase without spatial or temporal phase unwrapping. Unlike the conventional absolute 3D shape measurement techniques, the proposed method requires no spatial or temporal phase unwrapping. Therefore, the proposed method does not have the limitation of the spatial phase unwrapping method: “smooth” surface requirement; it also does not have the limitation associated with the temporal phase unwrapping: more patterns requirement that slows down the measurement speeds.
  • High speed. The proposed method combines the merits of the pseudo-random pattern based method with the phase-shifting-based method to recover absolute phase. The proposed method could achieve highest possible speed for a 3D shape measurement with a phase-shifting method since it requires the minimum number of phase-shifted fringe patterns (three) for pixel by pixel phase retrieval.
  • No nonlinear gamma correction requirement. Unlike the conventional phase-based method where the nonlinear gamma correction is usually required, the proposed method does not have to correct the nonlinear gamma of the projector.
  • No projector calibration requirement. The proposed method completely eliminates the requirement of projector calibration, which is usually not easy and difficult to achieve high accuracy for high-speed 3D shape measurement cases. Since no projector calibration is required, the projection system could be simplified (e.g., using an inexpensive projection engine, or even a simple slide projector).
  • No high-quality phase requirement. This paper demonstrates that high-quality 3D shape measurement can be realized even with very poor quality phase data, alleviating the stringent requirements of the conventional high-quality 3D shape measurement with phase-based methods. This could further simplify the 3D shape measurement system development since the projection noise or the camera noise will not substantially influence the measurement quality.

Yet, the proposed method also has some sacrifices. The major drawbacks associated with the proposed method are:
  • Low SNR. It reduces the fringe quality (SNR) by encoding the statistical pattern to the quality map, making it difficult to directly use phase maps to recover 3D shape without stereo cameras.
  • More computational cost. Comparing with the standard stereo-based method, or phase-based method, the computation time is quite significant due to the pixel-by-pixel disparity refinements besides the stereo matching time cost.
  • Large shade/occlusion areas. The proposed techniques uses the hardware systems includes two cameras and one projector. Theoretically, the system can be constructed into dual camera-projector, structured-light 3D shape system that could enlarge the measurement range in a similar manner to the one we have developed before [65

    65. S. Zhang and S.-T. Yau, “Three-dimensional shape measurement using a structured light system with dual cameras,” Opt. Eng. 471), 013,604 (2008). [CrossRef]

    ]. The proposed method, however, only performs 3D shape measurements where three sensors (two cameras and one projector) can all see, or the overlapping area of the 3D sensors, and thus the whole measurement areas are smaller than either structured light system.

Despite these shortcomings, the values of the proposed method are still substantial for high-speed and high-spatial resolution measurements without using expensive hardware components. This is because the proposed technique uses a phase-based method, and simultaneously alleviates the stringent requirements of the conventional phase-based methods.

5. Summary

This paper has presented a novel method for 3D absolute shape measurement without a traditional spatial or temporal phase unwrapping algorithm. We proposed to encode the quality map of the phase-shifted fringe patterns for rough disparity map determination by employing the ELAS algorithm, and the wrapped phase to refine the rough disparity map for high-quality 3D shape measurement. The proposed method also does not require any projector calibration, or high-quality phase map, and thus could potentially simplify the 3D shape measurement system development. Experimental results demonstrated the success of the proposed technique.

Acknowledgments

References and links

1.

D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Intl J. Comp. Vis. 47(1–3), 7–42 (2002). [CrossRef]

2.

U. R. Dhond and J. K. Aggarwal, “Structure from stereo-a review,” IEEE Trans. Systems, Man. and Cybernetics 19(6), 1489–1510 (1989). [CrossRef]

3.

R. I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge University Press, ISBN: 0521623049, 2000).

4.

T. Kanade and M. Okutomi, “A stereo matching algorithm with an adaptive window: Theory and experiment,” IEEE Trans. Patt. Analy. and Mach. Intellig. 16(9), 920–932 (1994). [CrossRef]

5.

V. Kolmogorov and R. Zabih, “Multi-camera scene reconstruction via graph cuts,” in Euro Conf. Comp. Vis., pp. 82–96 (2002).

6.

J. Kostková and R. Sára, “Stratified Dense Matching for Stereopsis in Complex Scenes.” in Proc. Brit. Mach. Vis. Conf., pp. 339–348 (2003).

7.

H. Hirschmuller, “Stereo processing by semiglobal matching and mutual information,” IEEE Trans. Patt. Analysis Mach. Intellig. 30(2), 328–341 (2008). [CrossRef]

8.

F. Besse, C. Rother, A. W. Fitzgibbon, and J. Kautz, “PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation,” Intl J. Comp. Vis. pp. 1–12 (Springer2013).

9.

A. Geiger, M. Roser, and R. Urtasun, “Efficient Large-Scale Stereo Matching,” 6492, 25–38 (2011).

10.

J. Salvi, S. Fernandez, T. Pribanic, and X. Llado, “A state of the art in structured light patterns for surface profilometry,” Patt. Recogn. 43(8), 2666–2680 (2010). [CrossRef]

11.

Z. Zhang, “Microsoft Kinect Sensor and Its Effect,” IEEE Multimedia 19(2), 4–10 (2012). [CrossRef]

12.

O. Hall-Holt and S. Rusinkiewicz, “Stripe Boundary Codes for Real-Time Structured-Light Range Scanning of Moving Objects,” in The 8th IEEE International Conference on Computer Vision, pp. II: 359–366 (2001).

13.

S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, “Real-time 3D Model Acquisition,” ACM Trans. Graph. 21(3), 438–446 (2002). [CrossRef]

14.

D. Malacara, ed., Optical Shop Testing, 3 (John Wiley and Sons, New York, NY, 2007). [CrossRef]

15.

S. Zhang, “Recent progresses on real-time 3-D shape measurement using digital fringe projection techniques,” Opt. Laser Eng. 48(2), 149–158 (2010). [CrossRef]

16.

G. Geng, “Structured-light 3D surface imaging: a tutorial,” Advances in Opt. and Photonics 3(2), 128–160 (2011). [CrossRef]

17.

S. Gorthi and P. Rastogi, “Fringe projection techniques: Whither we are?” Opt. Laser. Eng. 48, 133–140 (2010). [CrossRef]

18.

D. C. Ghiglia and M. D. Pritt, eds., Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (John Wiley and Sons, New York, 1998).

19.

K. Creath, “Step height measurement using two-wavelength phase-shifting interferometry,” Appl. Opt. 26(14), 2810–2816 (1987). [CrossRef] [PubMed]

20.

Y.-Y. Cheng and J. C. Wyant, “Multiple-Wavelength Phase Shifting Interferometry,” Appl. Opt. 24, 804–807 (1985). [CrossRef]

21.

G. Sansoni, M. Carocci, and R. Rodella, “Three-Dimensional Vision Based on a Combination of Gray-Code and Phase-Shift Light Projection: Analysis and Compensation of the Systematic Errors,” Appl. Opt. 38, 6565–6573 (1999). [CrossRef]

22.

U. R. Dhond and J. K. Aggarwal, “Real-time scalable depth sensing with hybrid structured light illumination,” IEEE Trans. Imag. Proc. 23(1), 97–109 (2013).

23.

S. Zhang, “Composite phase-shifting algorithm for absolute phase measurement,” Opt. Laser Eng. 50(11), 1538–1541 (2012). [CrossRef]

24.

X. Chen, J. Xi, Y. Jin, and J. Sun, “Accurate calibration for a camera–projector measurement system based on structured light projection,” Opt. Laser Eng. 47(3), 310–319 (2009). [CrossRef]

25.

Q. Hu, P. S. Huang, Q. Fu, and F. P. Chiang, “Calibration of a 3-D Shape Measurement System,” Opt. Eng. 42(2), 487–493 (2003). [CrossRef]

26.

F. J. Cuevas, M. Servin, and R. Rodriguez-Vera, “Depth Object Recovery Using Radial Basis Functions,” Opt. Commun. 163(4), 270–277 (1999). [CrossRef]

27.

R. Legarda-Sáenz, T. Bothe, and W. P. Jüptner, “Accurate Procedure for the Calibration of a Structured Light System,” Opt. Eng. 43(2), 464–471 (2004). [CrossRef]

28.

S. Zhang and P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng. 458), 083,601 (2006). [CrossRef]

29.

W. Gao, L. Wang, and Z. Hu, “Flexible method for structured light system calibration,” Opt. Eng. 478), 083,602 (2008). [CrossRef]

30.

R. Yang, S. Cheng, and Y. Chen, “Flexible and accurate implementation of a binocular structured light system,” Opt. Lasers Eng. 46(5), 373–379 (2008). [CrossRef]

31.

Z. Li, Y. Shi, C. Wang, and Y. Wang, “Accurate calibration method for a structured light system,” Opt. Eng. 475), 053,604 (2008). [CrossRef]

32.

S. Kakunai, T. Sakamoto, and K. Iwata, “Profile measurement taken with liquid-crystal grating,” Appl. Opt. 38(13), 2824–2828 (1999). [CrossRef]

33.

H. Guo, H. He, and M. Chen, “Gamma correction for digital fringe projection profilometry,” Appl. Opt. 43, 2906–2914 (2004). [CrossRef] [PubMed]

34.

B. Pan, Q. Kemao, L. Huang, and A. Asundi, “Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry,” Opt. Lett. 34(4), 2906–2914 (2009). [CrossRef]

35.

P. S. Huang, C. Zhang, and F.-P. Chiang, “High-speed 3-D Shape Measurement Based on Digital Fringe Projection,” Opt. Eng. 42(1), 163–168 (2002). [CrossRef]

36.

S. Zhang and P. S. Huang, “Phase error compensation for a three-dimensional shape measurement system based on the phase shifting method,” Opt. Eng. 466), 063,601 (2007). [CrossRef]

37.

S. Zhang and S.-T. Yau, “Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector,” Appl. Opt. 46(1), 36–43 (2007). [CrossRef]

38.

J. Pan, P. S. Huang, and F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng. 4512), 013,602 (2006). [CrossRef]

39.

J. Pan, P. S. Huang, and F.-P. Chiang, “Color-coded binary fringe projection technique for 3-D shape measurement,” Opt. Eng. 4412), 023,606 (2005). [CrossRef]

40.

C. Je, S. W. Lee, and R.-H. Park, “Colour-Stripe Permutation Pattern for Rapid Structured-Light Range Imaging,” Opt. Comm. 285(9), 2320–2331 (2012). [CrossRef]

41.

C. Je, S. W. Lee, and R.-H. Park, “Experimental comparison of phase-shifting fringe projection and statistical pattern projection for active triangulation systems,” in Proc. 6th Asian Conference on Computer Vision, pp. 270–275 (2004).

42.

Z. Zhang, C. E. Towers, and D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection,” Opt. Express 14(14), 6444–6455 (2006). [CrossRef] [PubMed]

43.

K. G. Harding, “Color Encoded Morié Contouring,” in Proc. SPIE , vol. 1005, pp. 169–178 (1988). [CrossRef]

44.

Z. J. Geng, “Rainbow 3-D Camera: New Concept of High-Speed Three Vision System,” Opt. Eng. 35, 376–383 (1996). [CrossRef]

45.

P. S. Huang, Q. Hu, F. Jin, and F. P. Chiang, “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring,” Opt. Eng. 38, 1065–1071 (1999). [CrossRef]

46.

S. Zhang and P. S. Huang, “High-resolution real-time three-dimensional shape measurement,” Opt. Eng. 4512), 123,601 (2006). [CrossRef]

47.

Y. Huang, Y. Shang, Y. Liu, and H. Bao, Handbook of 3D Machine Vision: Optical Metrology and Imaging, chap. 3D shapes from Speckle, pp. 33–56, 1 (CRC, 2013). [CrossRef]

48.

L. Zhang, N. Snavely, B. Curless, and S. M. Seitz, “Spacetime Faces: High-Resolution Capture for Modeling and Animation,” ACM Trans. Graph. 23(3), 548–558 (2004). [CrossRef]

49.

A. Wiegmann, H. Wagner, and R. Kowarschik, “Human face measurement by projecting bandlimited random patterns,” Opt. Express 14(17), 7692–7698 (2006). [CrossRef] [PubMed]

50.

J. Davis, R. Ramamoorthi, and S. Rusinkiewicz, “Spacetime Stereo: A Unifying Framework for Depth from Triangulation,” IEEE Trans. Patt. Anal. and Mach.e Intell. 27(2), 1–7 (2005).

51.

L. Zhang, B. Curless, and S. Seitz, “Spacetime Stereo: Shape Recovery for Dynamic Scenes,” in Proc. Comp. Vis. Patt. Recogn., pp. 367–374 (2003).

52.

W. Jang, C. Je, Y. Seo, and S. W. Lee, “Structured-light stereo: Comparative analysis and integration of structured-light and active stereo for measuring dynamic shape,” Opt. Laser Eng. 51(11), 1255–1264 (2013). [CrossRef]

53.

X. Han and P. Huang, “Combined stereovision and phase shifting method: use of a visibility-modulated fringe pattern,” in SPIE Europe Optical Metrology, pp. 73,893H (2009).

54.

M. Schaffer, M. Große, B. Harendt, and R. Kowarschik, “Coherent two-beam interference fringe projection for highspeed three-dimensional shape measurements,” Appl. Opt. 52(11), 2306–2311 (2013). [CrossRef] [PubMed]

55.

K. Liu and Y. Wang, “Phase channel multiplexing pattern strategy for active stereo vision,” in Intl Conf. 3D Imaging (IC3D), pp. 1–8 (2012). [CrossRef]

56.

J. Salvi, J. Pages, and J. Batlle, “Pattern codification strategies in structured light systems,” Patt. Recogn. 37(4), 827–849 (2004). [CrossRef]

57.

P. Lutzke, M. Schaffer, P. Kühmstedt, R. Kowarschik, and G. Notni, “Experimental comparison of phase-shifting fringe projection and statistical pattern projection for active triangulation systems,” in SPIE Optical Metrology 2013, pp. 878,813 (2013).

58.

C. Bräuer-Burchardt, M. Möller, C. Munkelt, M. Heinze, P. Kühmstedt, and G. Notni, “On the accuracy of point correspondence methods in three-dimensional measurement systems using fringe projection,” Opt. Eng. 526), 063,601 (2013). [CrossRef]

59.

Z. Li, K. Zhong, Y. Li, X. Zhou, and Y. Shi, “Multiview phase shifting: a full-resolution and high-speed 3D measurement framework for arbitrary shape dynamic objects,” Opt. Lett. 38(9), 1389–1391 (2013). [CrossRef] [PubMed]

60.

K. Zhong, Z. Li, Y. Shi, C. Wang, and Y. Lei, “Fast phase measurement profilometry for arbitrary shape objects without phase unwrapping,” Opt. Laser Eng. 51(11), 1213–1222 (2013). [CrossRef]

61.

C. Bräuer-Burchardt, P. Kühmstedt, and G. Notni, “Phase unwrapping using geometric constraints for high-speed fringe projection based 3D measurements,” in SPIE Optical Metrology 2013, pp. 878,906 (2013).

62.

M. Maruyama and S. Abe, “Range sensing by projecting multiple slits with random cuts,” IEEE Trans. Patt. Analysis Mach. Intellig. 15(6), 647–651 (1993). [CrossRef]

63.

K. Konolige, “Projected texture stereo,” inIEEE Intl Conf. Rob. Auto., pp. 148–155 (2010).

64.

Y. Wang and S. Zhang, “Superfast multifrequency phase-shifting technique with optimal pulse width modulation,” Opt. Express 19(6), 5143–5148 (2011).

65.

S. Zhang and S.-T. Yau, “Three-dimensional shape measurement using a structured light system with dual cameras,” Opt. Eng. 471), 013,604 (2008). [CrossRef]

OCIS Codes
(110.6880) Imaging systems : Three-dimensional image acquisition
(120.0120) Instrumentation, measurement, and metrology : Instrumentation, measurement, and metrology
(120.5050) Instrumentation, measurement, and metrology : Phase measurement
(320.7100) Ultrafast optics : Ultrafast measurements

ToC Category:
Instrumentation, Measurement, and Metrology

History
Original Manuscript: October 14, 2013
Revised Manuscript: December 24, 2013
Manuscript Accepted: January 3, 2014
Published: January 13, 2014

Citation
William Lohry, Vincent Chen, and Song Zhang, "Absolute three-dimensional shape measurement using coded fringe patterns without phase unwrapping or projector calibration," Opt. Express 22, 1287-1301 (2014)
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-22-2-1287


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. D. Scharstein, R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” Intl J. Comp. Vis. 47(1–3), 7–42 (2002). [CrossRef]
  2. U. R. Dhond, J. K. Aggarwal, “Structure from stereo-a review,” IEEE Trans. Systems, Man. and Cybernetics 19(6), 1489–1510 (1989). [CrossRef]
  3. R. I. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge University Press, ISBN: 0521623049, 2000).
  4. T. Kanade, M. Okutomi, “A stereo matching algorithm with an adaptive window: Theory and experiment,” IEEE Trans. Patt. Analy. and Mach. Intellig. 16(9), 920–932 (1994). [CrossRef]
  5. V. Kolmogorov, R. Zabih, “Multi-camera scene reconstruction via graph cuts,” in Euro Conf. Comp. Vis., pp. 82–96 (2002).
  6. J. Kostková, R. Sára, “Stratified Dense Matching for Stereopsis in Complex Scenes.” in Proc. Brit. Mach. Vis. Conf., pp. 339–348 (2003).
  7. H. Hirschmuller, “Stereo processing by semiglobal matching and mutual information,” IEEE Trans. Patt. Analysis Mach. Intellig. 30(2), 328–341 (2008). [CrossRef]
  8. F. Besse, C. Rother, A. W. Fitzgibbon, J. Kautz, “PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation,” Intl J. Comp. Vis. pp. 1–12 (Springer2013).
  9. A. Geiger, M. Roser, R. Urtasun, “Efficient Large-Scale Stereo Matching,” 6492, 25–38 (2011).
  10. J. Salvi, S. Fernandez, T. Pribanic, X. Llado, “A state of the art in structured light patterns for surface profilometry,” Patt. Recogn. 43(8), 2666–2680 (2010). [CrossRef]
  11. Z. Zhang, “Microsoft Kinect Sensor and Its Effect,” IEEE Multimedia 19(2), 4–10 (2012). [CrossRef]
  12. O. Hall-Holt, S. Rusinkiewicz, “Stripe Boundary Codes for Real-Time Structured-Light Range Scanning of Moving Objects,” in The 8th IEEE International Conference on Computer Vision, pp. II: 359–366 (2001).
  13. S. Rusinkiewicz, O. Hall-Holt, M. Levoy, “Real-time 3D Model Acquisition,” ACM Trans. Graph. 21(3), 438–446 (2002). [CrossRef]
  14. D. Malacara, ed., Optical Shop Testing, 3 (John Wiley and Sons, New York, NY, 2007). [CrossRef]
  15. S. Zhang, “Recent progresses on real-time 3-D shape measurement using digital fringe projection techniques,” Opt. Laser Eng. 48(2), 149–158 (2010). [CrossRef]
  16. G. Geng, “Structured-light 3D surface imaging: a tutorial,” Advances in Opt. and Photonics 3(2), 128–160 (2011). [CrossRef]
  17. S. Gorthi, P. Rastogi, “Fringe projection techniques: Whither we are?” Opt. Laser. Eng. 48, 133–140 (2010). [CrossRef]
  18. D. C. Ghiglia, M. D. Pritt, eds., Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software (John Wiley and Sons, New York, 1998).
  19. K. Creath, “Step height measurement using two-wavelength phase-shifting interferometry,” Appl. Opt. 26(14), 2810–2816 (1987). [CrossRef] [PubMed]
  20. Y.-Y. Cheng, J. C. Wyant, “Multiple-Wavelength Phase Shifting Interferometry,” Appl. Opt. 24, 804–807 (1985). [CrossRef]
  21. G. Sansoni, M. Carocci, R. Rodella, “Three-Dimensional Vision Based on a Combination of Gray-Code and Phase-Shift Light Projection: Analysis and Compensation of the Systematic Errors,” Appl. Opt. 38, 6565–6573 (1999). [CrossRef]
  22. U. R. Dhond, J. K. Aggarwal, “Real-time scalable depth sensing with hybrid structured light illumination,” IEEE Trans. Imag. Proc. 23(1), 97–109 (2013).
  23. S. Zhang, “Composite phase-shifting algorithm for absolute phase measurement,” Opt. Laser Eng. 50(11), 1538–1541 (2012). [CrossRef]
  24. X. Chen, J. Xi, Y. Jin, J. Sun, “Accurate calibration for a camera–projector measurement system based on structured light projection,” Opt. Laser Eng. 47(3), 310–319 (2009). [CrossRef]
  25. Q. Hu, P. S. Huang, Q. Fu, F. P. Chiang, “Calibration of a 3-D Shape Measurement System,” Opt. Eng. 42(2), 487–493 (2003). [CrossRef]
  26. F. J. Cuevas, M. Servin, R. Rodriguez-Vera, “Depth Object Recovery Using Radial Basis Functions,” Opt. Commun. 163(4), 270–277 (1999). [CrossRef]
  27. R. Legarda-Sáenz, T. Bothe, W. P. Jüptner, “Accurate Procedure for the Calibration of a Structured Light System,” Opt. Eng. 43(2), 464–471 (2004). [CrossRef]
  28. S. Zhang, P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng. 458), 083,601 (2006). [CrossRef]
  29. W. Gao, L. Wang, Z. Hu, “Flexible method for structured light system calibration,” Opt. Eng. 478), 083,602 (2008). [CrossRef]
  30. R. Yang, S. Cheng, Y. Chen, “Flexible and accurate implementation of a binocular structured light system,” Opt. Lasers Eng. 46(5), 373–379 (2008). [CrossRef]
  31. Z. Li, Y. Shi, C. Wang, Y. Wang, “Accurate calibration method for a structured light system,” Opt. Eng. 475), 053,604 (2008). [CrossRef]
  32. S. Kakunai, T. Sakamoto, K. Iwata, “Profile measurement taken with liquid-crystal grating,” Appl. Opt. 38(13), 2824–2828 (1999). [CrossRef]
  33. H. Guo, H. He, M. Chen, “Gamma correction for digital fringe projection profilometry,” Appl. Opt. 43, 2906–2914 (2004). [CrossRef] [PubMed]
  34. B. Pan, Q. Kemao, L. Huang, A. Asundi, “Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry,” Opt. Lett. 34(4), 2906–2914 (2009). [CrossRef]
  35. P. S. Huang, C. Zhang, F.-P. Chiang, “High-speed 3-D Shape Measurement Based on Digital Fringe Projection,” Opt. Eng. 42(1), 163–168 (2002). [CrossRef]
  36. S. Zhang, P. S. Huang, “Phase error compensation for a three-dimensional shape measurement system based on the phase shifting method,” Opt. Eng. 466), 063,601 (2007). [CrossRef]
  37. S. Zhang, S.-T. Yau, “Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector,” Appl. Opt. 46(1), 36–43 (2007). [CrossRef]
  38. J. Pan, P. S. Huang, F.-P. Chiang, “Color phase-shifting technique for three-dimensional shape measurement,” Opt. Eng. 4512), 013,602 (2006). [CrossRef]
  39. J. Pan, P. S. Huang, F.-P. Chiang, “Color-coded binary fringe projection technique for 3-D shape measurement,” Opt. Eng. 4412), 023,606 (2005). [CrossRef]
  40. C. Je, S. W. Lee, R.-H. Park, “Colour-Stripe Permutation Pattern for Rapid Structured-Light Range Imaging,” Opt. Comm. 285(9), 2320–2331 (2012). [CrossRef]
  41. C. Je, S. W. Lee, R.-H. Park, “Experimental comparison of phase-shifting fringe projection and statistical pattern projection for active triangulation systems,” in Proc. 6th Asian Conference on Computer Vision, pp. 270–275 (2004).
  42. Z. Zhang, C. E. Towers, D. P. Towers, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection,” Opt. Express 14(14), 6444–6455 (2006). [CrossRef] [PubMed]
  43. K. G. Harding, “Color Encoded Morié Contouring,” in Proc. SPIE, vol. 1005, pp. 169–178 (1988). [CrossRef]
  44. Z. J. Geng, “Rainbow 3-D Camera: New Concept of High-Speed Three Vision System,” Opt. Eng. 35, 376–383 (1996). [CrossRef]
  45. P. S. Huang, Q. Hu, F. Jin, F. P. Chiang, “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring,” Opt. Eng. 38, 1065–1071 (1999). [CrossRef]
  46. S. Zhang, P. S. Huang, “High-resolution real-time three-dimensional shape measurement,” Opt. Eng. 4512), 123,601 (2006). [CrossRef]
  47. Y. Huang, Y. Shang, Y. Liu, H. Bao, Handbook of 3D Machine Vision: Optical Metrology and Imaging, chap. 3D shapes from Speckle, pp. 33–56, 1 (CRC, 2013). [CrossRef]
  48. L. Zhang, N. Snavely, B. Curless, S. M. Seitz, “Spacetime Faces: High-Resolution Capture for Modeling and Animation,” ACM Trans. Graph. 23(3), 548–558 (2004). [CrossRef]
  49. A. Wiegmann, H. Wagner, R. Kowarschik, “Human face measurement by projecting bandlimited random patterns,” Opt. Express 14(17), 7692–7698 (2006). [CrossRef] [PubMed]
  50. J. Davis, R. Ramamoorthi, S. Rusinkiewicz, “Spacetime Stereo: A Unifying Framework for Depth from Triangulation,” IEEE Trans. Patt. Anal. and Mach.e Intell. 27(2), 1–7 (2005).
  51. L. Zhang, B. Curless, S. Seitz, “Spacetime Stereo: Shape Recovery for Dynamic Scenes,” in Proc. Comp. Vis. Patt. Recogn., pp. 367–374 (2003).
  52. W. Jang, C. Je, Y. Seo, S. W. Lee, “Structured-light stereo: Comparative analysis and integration of structured-light and active stereo for measuring dynamic shape,” Opt. Laser Eng. 51(11), 1255–1264 (2013). [CrossRef]
  53. X. Han, P. Huang, “Combined stereovision and phase shifting method: use of a visibility-modulated fringe pattern,” in SPIE Europe Optical Metrology, pp. 73,893H (2009).
  54. M. Schaffer, M. Große, B. Harendt, R. Kowarschik, “Coherent two-beam interference fringe projection for highspeed three-dimensional shape measurements,” Appl. Opt. 52(11), 2306–2311 (2013). [CrossRef] [PubMed]
  55. K. Liu, Y. Wang, “Phase channel multiplexing pattern strategy for active stereo vision,” in Intl Conf. 3D Imaging (IC3D), pp. 1–8 (2012). [CrossRef]
  56. J. Salvi, J. Pages, J. Batlle, “Pattern codification strategies in structured light systems,” Patt. Recogn. 37(4), 827–849 (2004). [CrossRef]
  57. P. Lutzke, M. Schaffer, P. Kühmstedt, R. Kowarschik, G. Notni, “Experimental comparison of phase-shifting fringe projection and statistical pattern projection for active triangulation systems,” in SPIE Optical Metrology 2013, pp. 878,813 (2013).
  58. C. Bräuer-Burchardt, M. Möller, C. Munkelt, M. Heinze, P. Kühmstedt, G. Notni, “On the accuracy of point correspondence methods in three-dimensional measurement systems using fringe projection,” Opt. Eng. 526), 063,601 (2013). [CrossRef]
  59. Z. Li, K. Zhong, Y. Li, X. Zhou, Y. Shi, “Multiview phase shifting: a full-resolution and high-speed 3D measurement framework for arbitrary shape dynamic objects,” Opt. Lett. 38(9), 1389–1391 (2013). [CrossRef] [PubMed]
  60. K. Zhong, Z. Li, Y. Shi, C. Wang, Y. Lei, “Fast phase measurement profilometry for arbitrary shape objects without phase unwrapping,” Opt. Laser Eng. 51(11), 1213–1222 (2013). [CrossRef]
  61. C. Bräuer-Burchardt, P. Kühmstedt, G. Notni, “Phase unwrapping using geometric constraints for high-speed fringe projection based 3D measurements,” in SPIE Optical Metrology 2013, pp. 878,906 (2013).
  62. M. Maruyama, S. Abe, “Range sensing by projecting multiple slits with random cuts,” IEEE Trans. Patt. Analysis Mach. Intellig. 15(6), 647–651 (1993). [CrossRef]
  63. K. Konolige, “Projected texture stereo,” inIEEE Intl Conf. Rob. Auto., pp. 148–155 (2010).
  64. Y. Wang, S. Zhang, “Superfast multifrequency phase-shifting technique with optimal pulse width modulation,” Opt. Express 19(6), 5143–5148 (2011).
  65. S. Zhang, S.-T. Yau, “Three-dimensional shape measurement using a structured light system with dual cameras,” Opt. Eng. 471), 013,604 (2008). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited