## Composite structured light pattern for three-dimensional video

Optics Express, Vol. 11, Issue 5, pp. 406-417 (2003)

http://dx.doi.org/10.1364/OE.11.000406

Acrobat PDF (1250 KB)

### Abstract

Based on recent discoveries, we introduce a method to project a single structured pattern onto an object and then reconstruct the three-dimensional range from the distortions in the reflected and captured image. Traditional structured light methods require several different patterns to recover the depth, without ambiguity or albedo sensitivity, and are corrupted by object movement during the projection/capture process. Our method efficiently combines multiple patterns into a single composite pattern projection allowing for real-time implementations. Because structured light techniques require standard image capture and projection technology, unlike time of arrival techniques, they are relatively low cost.

© 2002 Optical Society of America

## 1. Introduction

1. G. Schmaltz of Schmaltz Brothers Laboratories, “A method for presenting the profile curves of rough surfaces,” Naturwiss **18**, 315–316 (1932). [CrossRef]

2. P. M. Will and K. S. Pennington, “Grid coding: A preprocessing technique for robot and machine vision,” Artif. Intell. **2**, 319–329 (1971). [CrossRef]

*i.e.*, albedo), and/or they suffer from lower lateral resolution caused by the required spacing between stripes [3

3. R. C. Daley and L. G. Hassebrook, “Channel capacity model of binary encoded structured lightstripe illumination,” Appl. Opt. **37**, 3689–3696 (1998). [CrossRef]

4. J. L. Posdamer and M. D. Altschuler, “Surface measurement by space-encoded projected beam systems,” Comput. Vision Graph. Image Process. **18**, 1–17 (1982). [CrossRef]

5. V. Srinivasan, H. C. Liu, and M. Halioua, “Automated phase measuring profilometry: a phase mapping approach,” Appl. Opt. **24**, 185–188 (1985). [CrossRef] [PubMed]

7. Q. Fang and S. Zheng, “Linearly coded profilometry,” Appl. Opt. **36**, 2401–2407 (1997). [CrossRef] [PubMed]

9. O. A. Skydan, M. J. Lalor, and D. R. Burton, “Technique for phase measurement and surface reconstruction by use of colored structured light,” Appl. Opt. **41**, 6104–6117 (2002). [CrossRef] [PubMed]

11. B. Carrihill and R. Hummel, “Experiments with intensity ratio depth sensor,” Comput. Vision Graph. Image Process. **32**, 337–358 (1985). [CrossRef]

12. M. Maruyama and S. Abe, “Range sensing by projecting multiple slits with random cuts,” IEEE Trans. Pattern. Anal. Mach. Intell. **15**, 647–651 (1993). [CrossRef]

11. B. Carrihill and R. Hummel, “Experiments with intensity ratio depth sensor,” Comput. Vision Graph. Image Process. **32**, 337–358 (1985). [CrossRef]

12. M. Maruyama and S. Abe, “Range sensing by projecting multiple slits with random cuts,” IEEE Trans. Pattern. Anal. Mach. Intell. **15**, 647–651 (1993). [CrossRef]

3. R. C. Daley and L. G. Hassebrook, “Channel capacity model of binary encoded structured lightstripe illumination,” Appl. Opt. **37**, 3689–3696 (1998). [CrossRef]

*i.e.*, orthogonal dimension) to the depth distortion (

*i.e.*, phase dimension) was underutilized and could be used to modulate and combine multiple patterns into a single composite pattern [14]. Furthermore, this is a methodology that can be applied to a variety of existing multi-pattern techniques.

*ad hoc*single pattern techniques mentioned above, we introduce a systematic methodology to combine multiple patterns into one single composite pattern, based on well-known communications theory. The individual patterns are spatially modulated along the orthogonal dimension, perpendicular to the phase dimension. In this way we can then take advantage of the existing procedure for traditional multiple patterns such as Phase Measuring Profilometry (PMP) [5

5. V. Srinivasan, H. C. Liu, and M. Halioua, “Automated phase measuring profilometry: a phase mapping approach,” Appl. Opt. **24**, 185–188 (1985). [CrossRef] [PubMed]

7. Q. Fang and S. Zheng, “Linearly coded profilometry,” Appl. Opt. **36**, 2401–2407 (1997). [CrossRef] [PubMed]

15. J. Batlle, E. Mouaddib, and J. Salvi, “Recent progress in coded structured light as a technique to solve the correspondence problem: A survey,” Pattern Recogn. **31**, 963–982 (1998). [CrossRef]

16. F. Chen, G. M. Brown, and M. Song, “Overview of three-dimensional shape measurement using optical methods,” Opt. Eng. **39**, 10–22 (2000). [CrossRef]

## 2. Traditional PMP method

*N*for

*N*times as

*A*

^{p}and

*B*

^{p}are the projection constants and (

*x*

^{p}

*, y*

^{p}) is the projector coordinates. The

*y*

^{p}dimension is in the direction of the depth distortion and is called the

*phase dimension*. On the other hand,

*x*

^{p}dimension is perpendicular to the phase dimension, so we call it the

*orthogonal dimension*. The frequency

*f*ϕ of the sinusoid wave is in the phase direction. The subscript

*n*represents the phase shift index and

*n*=1, 2, …,

*N*, where

*N*is the total number of phase shifts.

*x, y*) are the image coordinates and α(

*x, y*) is the reflectance variation or the albedo. The pixel-wise phase distortion ϕ(

*x, y*) of the sinusoid wave corresponds to the object surface depth. The value of ϕ(

*x, y*) is determined from the captured patterns by

*x, y*), is cancelled in this calculation, therefore, the depth through this approach is independent of the albedo.

_{r}(

*x, y*) is pre-calculated from the projections on the reference plane. The depth of the object surface with respect to the reference plane is easily obtained through simple geometric algorithms [17

17. J. L. Li, H. J. Su, and X. Y. Su, “Two-frequency grating used in phase-measuring profilometry,” Appl. Opt. **36**, 277–280 (1997). [CrossRef] [PubMed]

*O*

_{p}, to the camera lens center,

*O*

_{c}, is

*d*. Both the projector and the projectorcamera plane are a distance

*L*from the reference plane. The height of the object at point

*A*,

*h*, is calculated by

*B̄C*is proportional to the difference between the phase at point

*B*, ϕ

_{B}, and the phase at point

*C*, ϕ

_{C}, as

*L*and

*d*, are determined during the calibration procedure.

18. T. R. Judge and P. J. Bryanston-Cross, “A review of phase unwrapping techniques in fringe analysis,” Opt. Lasers Eng. **21**, 199–239 (1994). [CrossRef]

19. H. Zhao, W. Chen, and Y. Tan, “Phase-unwrapping algorithm for the measurement of threedimensional object shapes,” Appl. Opt. **33**, 4497–4500 (1994). [CrossRef] [PubMed]

20. J. Li, L.G. Hassebrook, and Chun Guan, “Optimized two-frequency phase measuring profilometry light sensor temporal noise sensitivity,” J. Opt. Soc. Am. A **20**, 106–115 (2003). [CrossRef]

## 3. Composite PMP Pattern

*c*is used here to offset

*n*is the shift index from 1 to

*N*. The projection constants

*A*

^{p}and

*B*

^{p}are carefully calculated as

*I*

_{min},

*I*

_{max}]. In order to increase the SNR,

*B*

^{p}should reach its maximum value allowed [20

20. J. Li, L.G. Hassebrook, and Chun Guan, “Optimized two-frequency phase measuring profilometry light sensor temporal noise sensitivity,” J. Opt. Soc. Am. A **20**, 106–115 (2003). [CrossRef]

*I*

_{min},

*I*

_{max}] should match the intensity capacity of the projector to retrieve optimal depth information.

*f*

_{n}and are all derived from the same low-pass Butterworth filter design, in other words; they all have the uniform passband span and are symmetric at

*f*

_{n}. The Butterworth filter is used in this stage for smoother transition and minimal side-lobe ripple effect. On the other hand, the order of the Butterworth filter is carefully selected to reduce the crosstalk between channels. Compromising between side-lobe effects and crosstalk is required to obtain acceptable reconstruction performance. Cutoff frequencies for each band are designed such that

*n*=1, 2, 3, …,

*N*and

*f*

_{0}=0, which is the baseband channel. The orthogonal signal vectors after 1-D band-pass filtering are

*x*) is the band-pass filter along orthogonal direction centered at frequency

*f*

_{n}. The baseband image,

*I*’

_{n}(

*x*,

*y*), is assumed to be band limited along the orthogonal dimension with a bandwidth less than or equal to the filter

*x*) bandwidth.

*I*’

_{n}(

*x*,

*y*). Two critical practical factors have to be considered in the demodulation process. First, the perspective distortion causes the depth dependent variation of orthogonal carrier frequencies. Second, with the practical experimental setup, the cosine carrier wave on each orthogonal line has an unknown phase shift. That is, considering the perspective distortion, the practical image after band-pass filtering is based on Eq. (13) such that

*f*

_{n}has the small variation δ

*f*and δθ is the unknown phase shift. By squaring both sides of Eq. (14) we have

*h*’

_{LP}(

*x*) with a cutoff of ƒ

_{n}such that

*x*,

*y*) has to be non-negative. It is effectively an AM based modulation technique which recovers the PMP pattern as the positive envelope. The demodulation procedure is summarized in the diagram as in Fig. 4. The recovered images,

*x*,

*y*), represent the individual patterns in traditional PMP and are used to retrieve the depth of the measured object based on the traditional PMP method.

## 4. Experiments

*N*=4. The choice of

*N*=4 came from trial and error where the minimum of

*N*=3 has too much inherent reconstruction noise and

*N*>4 reduced the lateral resolution for the given camera resolution. In this experiment, carrier frequencies of the projector

## 5. Conclusion

## Acknowledgements

## References and links

1. | G. Schmaltz of Schmaltz Brothers Laboratories, “A method for presenting the profile curves of rough surfaces,” Naturwiss |

2. | P. M. Will and K. S. Pennington, “Grid coding: A preprocessing technique for robot and machine vision,” Artif. Intell. |

3. | R. C. Daley and L. G. Hassebrook, “Channel capacity model of binary encoded structured lightstripe illumination,” Appl. Opt. |

4. | J. L. Posdamer and M. D. Altschuler, “Surface measurement by space-encoded projected beam systems,” Comput. Vision Graph. Image Process. |

5. | V. Srinivasan, H. C. Liu, and M. Halioua, “Automated phase measuring profilometry: a phase mapping approach,” Appl. Opt. |

6. | Jielin Li and L. G. Hassebrook, “A robust svd based calibration of active range sensors,” in SPIE Proceedings on Visual Information Processing IX, S. K. Park and Z. Rahman, eds., (2000). |

7. | Q. Fang and S. Zheng, “Linearly coded profilometry,” Appl. Opt. |

8. | K. Boyer and A. Kak, “Color-encoded structured light for rapid active ranging,” IEEE Trans. Pattern. Anal. Mach. Intell. |

9. | O. A. Skydan, M. J. Lalor, and D. R. Burton, “Technique for phase measurement and surface reconstruction by use of colored structured light,” Appl. Opt. |

10. | D. S. Goodman and L. G. Hassebrook, “Face recognition under varying pose,” IBM Technical Disclosure Bulletin |

11. | B. Carrihill and R. Hummel, “Experiments with intensity ratio depth sensor,” Comput. Vision Graph. Image Process. |

12. | M. Maruyama and S. Abe, “Range sensing by projecting multiple slits with random cuts,” IEEE Trans. Pattern. Anal. Mach. Intell. |

13. | L. G. Hassebrook, R. C. Daley, and W. Chimitt, “Application of communication theory to high speed structured light illumination,” in SPIE Proceedings, Harding and Svetkoff, eds., Proc. |

14. | G. Goli, C. Guan, L. G. Hassebrook, and D. L. Lau, “Video rate three dimensional data acquisition using composite light structure pattern,” Tech. Rep. CSP 02-002, University of Kentucky, Department of Electrical and Computer Engineering, Lexington, KY USA (2002). |

15. | J. Batlle, E. Mouaddib, and J. Salvi, “Recent progress in coded structured light as a technique to solve the correspondence problem: A survey,” Pattern Recogn. |

16. | F. Chen, G. M. Brown, and M. Song, “Overview of three-dimensional shape measurement using optical methods,” Opt. Eng. |

17. | J. L. Li, H. J. Su, and X. Y. Su, “Two-frequency grating used in phase-measuring profilometry,” Appl. Opt. |

18. | T. R. Judge and P. J. Bryanston-Cross, “A review of phase unwrapping techniques in fringe analysis,” Opt. Lasers Eng. |

19. | H. Zhao, W. Chen, and Y. Tan, “Phase-unwrapping algorithm for the measurement of threedimensional object shapes,” Appl. Opt. |

20. | J. Li, L.G. Hassebrook, and Chun Guan, “Optimized two-frequency phase measuring profilometry light sensor temporal noise sensitivity,” J. Opt. Soc. Am. A |

**OCIS Codes**

(110.6880) Imaging systems : Three-dimensional image acquisition

(120.3180) Instrumentation, measurement, and metrology : Interferometry

**ToC Category:**

Research Papers

**History**

Original Manuscript: December 2, 2002

Revised Manuscript: February 19, 2003

Published: March 10, 2003

**Citation**

C. Guan, L. Hassebrook, and D. Lau, "Composite structured light pattern for three-dimensional video," Opt. Express **11**, 406-417 (2003)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-11-5-406

Sort: Journal | Reset

### References

- G. Schmaltz of Schmaltz Brothers Laboratories, �??A method for presenting the profile curves of rough surfaces,�?? Naturwiss 18, 315�??316 (1932). [CrossRef]
- P. M. Will and K. S. Pennington, �??Grid coding: A preprocessing technique for robot and machine vision,�?? Artif. Intell. 2, 319�??329 (1971). [CrossRef]
- R. C. Daley and L. G. Hassebrook, �??Channel capacity model of binary encoded structured lightstripe illumination,�?? Appl. Opt. 37, 3689�??3696 (1998). [CrossRef]
- J. L. Posdamer and M. D. Altschuler, �??Surface measurement by space-encoded projected beam systems,�?? Comput. Vision Graph. Image Process. 18, 1�??17 (1982). [CrossRef]
- V. Srinivasan, H. C. Liu, and M. Halioua, �??Automated phase measuring profilometry: a phase mapping approach,�?? Appl. Opt. 24, 185�??188 (1985). [CrossRef] [PubMed]
- Jielin Li and L. G. Hassebrook, �??A robust svd based calibration of active range sensors,�?? in SPIE Proceedings on Visual Information Processing IX, S. K. Park and Z. Rahman, eds., (2000).
- Q. Fang and S. Zheng, �??Linearly coded profilometry,�?? Appl. Opt. 36, 2401�??2407 (1997). [CrossRef] [PubMed]
- K. Boyer and A. Kak, �??Color-encoded structured light for rapid active ranging,�?? IEEE Trans. Pattern. Anal. Mach. Intell. 9, 2724�??2729 (1991).
- O. A. Skydan, M. J. Lalor, and D. R. Burton, �??Technique for phase measurement and surface reconstruction by use of colored structured light,�?? Appl. Opt. 41, 6104�??6117 (2002). [CrossRef] [PubMed]
- D. S. Goodman and L. G. Hassebrook, �??Face recognition under varying pose,�?? IBM Technical Disclosure Bulletin 27, 2671�??2673 (1984).
- B. Carrihill and R. Hummel, �??Experiments with intensity ratio depth sensor,�?? Comput. Vision Graph. Image Process. 32, 337�??358 (1985). [CrossRef]
- M. Maruyama and S. Abe, �??Range sensing by projecting multiple slits with random cuts,�?? IEEE Trans. Pattern. Anal. Mach. Intell. 15, 647�??651 (1993). [CrossRef]
- L. G. Hassebrook, R. C. Daley, and W. Chimitt, �??Application of communication theory to high speed structured light illumination,�?? in SPIE Proceedings, Harding and Svetko., eds., Proc. 3204(15), 102�??113 (1997)
- G. Goli, C. Guan, L. G. Hassebrook, and D. L. Lau, �??Video rate three dimensional data acquisition using composite light structure pattern,�?? Tech. Rep. CSP 02-002, University of Kentucky, Department of Electrical and Computer Engineering, Lexington, KY USA (2002).
- J. Batlle, E. Mouaddib, and J. Salvi, �??Recent progress in coded structured light as a technique to solve the correspondence problem: A survey,�?? Pattern Recogn. 31, 963�??982 (1998) [CrossRef]
- F. Chen, G. M. Brown, and M. Song, �??Overview of three-dimensional shape measurement using optical methods,�?? Opt. Eng. 39, 10�??22 (2000). [CrossRef]
- J. L. Li, H. J. Su, and X. Y. Su, �??Two-frequency grating used in phase-measuring profilometry,�?? Appl. Opt. 36, 277�??280 (1997). [CrossRef] [PubMed]
- T. R. Judge and P. J. Bryanston-Cross, �??A review of phase unwrapping techniques in fringe analysis,�?? Opt. Lasers Eng. 21, 199�??239 (1994). [CrossRef]
- H. Zhao, W. Chen, and Y. Tan, �??Phase-unwrapping algorithm for the measurement of threedimensional object shapes,�?? Appl. Opt. 33, 4497�??4500 (1994). [CrossRef] [PubMed]
- J. Li, L.G. Hassebrook, and Chun Guan, �??Optimized two-frequency phase measuring pro.lometry light sensor temporal noise sensitivity,�?? J. Opt. Soc. Am. A 20, 106�??115 (2003). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

### Supplementary Material

» Media 1: MPG (1036 KB)

» Media 2: MPG (1016 KB)

» Media 3: MPG (584 KB)

» Media 4: MPG (1886 KB)

OSA is a member of CrossRef.