## Snap-shot profilometry with the Empirical Mode Decomposition and a 3-layer color sensor |

Optics Express, Vol. 19, Issue 2, pp. 1284-1290 (2011)

http://dx.doi.org/10.1364/OE.19.001284

Acrobat PDF (1031 KB)

### Abstract

Remote sensing finds more and more applications, from industrial control, to face recognition, not forgetting terrain surveying. This trend is well exemplified by fringe projection techniques, which enjoyed a considerable development in the recent years. In addition of high requirement in terms of measurement accuracy and spatial resolution, the end-users of full-field techniques show a growing interest for dynamic regimes. We report here what we believe to be the use for the first time of a CMOS 3-layer color sensor (Foveon X3) as the key element of a RGB fringe projection system, together with the processing specifically elaborated for this sensor. The 3-layer architecture allows the simultaneous recording of three phase-shifted fringe patterns and features the precious asset of an unambiguous relationship between the physical sensor pixel and the picture pixel and this for each color layer, on the contrary of common color sensor arrays (Bayer mosaic and tri-CCD). Due to the overlapping of the spectral responses of the layers, color transformation is mandatory to achieve the separation of the three phase-shifted RGB projected fringe patterns. In addition, we propose the use of the Empirical Mode Decomposition to equalize the non-uniform responses of the three layers. Although the conversion of the phase into a height is of primary importance in an actual measurement, it is not treated here, the literature being profuse on the central projection model.

© 2011 OSA

## 1. Introduction

1. F. Chen, G. M. Brown, and M. Song, “Overview of three-dimensional shape measurement using optical methods,” Opt. Eng. **39**(1), 10–22 (2000). [CrossRef]

4. P. S. Huang, C. Zhang, and F.-P. Chiang, “High-speed 3-D shape measurement based on digital fringe projection,” Opt. Eng. **42**(1), 163–168 (2003). [CrossRef]

9. J. Novak, P. Novak, and A. Miks, “Multi-step phase-shifting algorithms insensitive to linear phase shift errors,” Opt. Commun. **281**(21), 5302–5309 (2008). [CrossRef]

10. X. Colonna de Lega and P. Jacquot, “Deformation measurement with object-induced dynamic phase shifting,” Appl. Opt. **35**(25), 5115–5121 (1996). [CrossRef] [PubMed]

11. C. Wust and D. W. Capson, “Surface profile measurement using color fringe projection,” Mach. Vis. Appl. **4**(3), 193–203 (1991). [CrossRef]

## 2. Layered CMOS camera

13. P. Tankam, P. Picart, D. Mounier, J. M. Desse, and J. C. Li, “Method of digital holographic recording and reconstruction using a stacked color image sensor,” Appl. Opt. **49**(3), 320–328 (2010). [CrossRef] [PubMed]

## 3. Fringe projection and phase computation

*s*, would obey the following generic equations:In parallel projection and telecentric observation (see Fig. 3 ), with fringes parallel to the

_{R,G,B}*y*axis, projected in a direction making an angle

*α*with the

*z*-axis, and of apparent period

*p*in the

*z*-observation direction, the relationship between

*φ*and

*h*is simply:where

*φ*accounts for the position of the projected fringes with respect to the origin of the coordinates.

_{0}## 4. Linearization and color transformation

16. J. A. N. Buytaert and J. J. J. Dirckx, “Phase-shifting moiré topography using optical demodulation on liquid crystal matrices,” Opt. Lasers Eng. **48**(2), 172–181 (2010). [CrossRef]

*X*, and the matrix

_{k}*T*of Eq. (5) is thus obtained by solving an over-determined system of 72 equations. The raw image grabbed by the camera is shown in Fig. 5 , jointly with its linearized dark-subtracted counterpart and the final image resulting of the color transformation.

## 5. The Empirical Mode Decomposition

18. N. E. Huang, Z. Shen, S. R. Long, M. C. Wu, H. H. Shih, Q. Zheng, N.-C. Yen, C. C. Tung, and H. H. Liu, “The empirical mode decomposition and the Hilbert spectrum for nonlinear and nonstationary time series analysis,” Proc. R. Soc. Lond. A **454**(1971), 903–995 (1998). [CrossRef]

*s(t)*constituted by a detail part (local high frequency)

*d(t)*, and a residue part (local low frequency)

*m(t)*. The detail part is sifted out from the raw signal by removing the mean envelope, whose computation is based on a cubic spline fitting between the signal extrema. The interpolation with a cubic spline kernel is acknowledged to have the best performances in most cases. The residue is then considered itself as a signal to process and thus split into a detail and a residue part as well. The final algorithm [19

19. P. Flandrin, http://perso.ens-lyon.fr/patrick.flandrin/emd.html.

*d*are the extracted IMFs and

_{k}(t)*m*is the final residue. The decomposition can be stopped at any rank K depending on the purpose, or in other words, depending on the frequency band the sought-after information belongs to (detrending, denoising, texture extraction and so on). If a total decomposition is wished, the procedure is ended when the current residue contains less than three extrema.

_{K}(t)20. S. Equis, “Phase extraction of non-stationary signals produced in dynamic interferometry involving speckle waves,” EPFL thesis n° **4514**, Lausanne, (2009). http://biblion.epfl.ch/EPFL/theses/2009/4514/EPFL_TH4514.pdf.

22. S. Equis and P. Jacquot, “The empirical mode decomposition: a must-have tool in speckle interferometry?” Opt. Express **17**(2), 611–623 (2009). [CrossRef] [PubMed]

20. S. Equis, “Phase extraction of non-stationary signals produced in dynamic interferometry involving speckle waves,” EPFL thesis n° **4514**, Lausanne, (2009). http://biblion.epfl.ch/EPFL/theses/2009/4514/EPFL_TH4514.pdf.

^{2}) with a cushion shape is illuminated with a tricolor fringe pattern. Figure 7 illustrates the way the EMD works on one portion of a row of the area of interest. The considered line is emphasized by a thick red line. The raw profiles from the three layers are shown in the top right graph, while the fully-processed signals − as a reminder, after linearization, dark-frame subtraction, color transformation and EMD − are shown below.

## 6. Experimental results, discussion and outlooks

23. H. A. Aebischer and S. Waldner, “Simple and effective method for filtering speckle interferometric phase fringe patterns,” Opt. Commun. **162**(4-6), 205–210 (1999). [CrossRef]

10. X. Colonna de Lega and P. Jacquot, “Deformation measurement with object-induced dynamic phase shifting,” Appl. Opt. **35**(25), 5115–5121 (1996). [CrossRef] [PubMed]

*i)*to know whether, in addition of being three times faster, the 3-layer color sensor and its dedicated processing is more accurate than the standard 3-image phase-shifting procedure based on three time-separated acquisitions,

*ii)*to discuss up to which extent the system can cope with color objects. Our first experiments do not show at least any evident discrepancies between the phase maps obtained with the new method and those produced, everything else kept as far as possible identical, by a black&white CCD camera or by the Foveon camera used sequentially and separately in each of its three layers. As for the second concern, we observed at the qualitative stage that color objects can be measured as well. A quite obvious condition must however be fulfilled: the object under analysis must not contain colors with one null RGB coordinate, so that after color-transformation, information remains from each phase-shifted pattern.

## Acknowledgments

## References and links

1. | F. Chen, G. M. Brown, and M. Song, “Overview of three-dimensional shape measurement using optical methods,” Opt. Eng. |

2. | E. Stoykova, A. A. Alatan, P. Benzie, N. Grammalidis, S. Malassiotis, J. Ostermann, S. Piekh, V. Sainov, C. Theobalt, T. Thevar, and X. Zabulis, “3-D time varying scene capture technologies – a survey,” IEEE Trans. Circ. Syst. Video Tech. |

3. | S. Zhang, “Recent progresses on real-time 3D shape measurement using digital fringe projection techniques,” Opt. Laser Technol. |

4. | P. S. Huang, C. Zhang, and F.-P. Chiang, “High-speed 3-D shape measurement based on digital fringe projection,” Opt. Eng. |

5. | K. Creath, |

6. | P. Groot, “Derivation of algorithms for phase-shifting interferometry using the concept of a data-sampling window,” Appl. Opt. |

7. | D. W. Phillion, “General methods for generating phase-shifting interferometry algorithms,” Appl. Opt. |

8. | Y. Surrel, “Design of algorithms for phase measurements by the use of phase stepping,” Appl. Opt. |

9. | J. Novak, P. Novak, and A. Miks, “Multi-step phase-shifting algorithms insensitive to linear phase shift errors,” Opt. Commun. |

10. | X. Colonna de Lega and P. Jacquot, “Deformation measurement with object-induced dynamic phase shifting,” Appl. Opt. |

11. | C. Wust and D. W. Capson, “Surface profile measurement using color fringe projection,” Mach. Vis. Appl. |

12. | |

13. | P. Tankam, P. Picart, D. Mounier, J. M. Desse, and J. C. Li, “Method of digital holographic recording and reconstruction using a stacked color image sensor,” Appl. Opt. |

14. | |

15. | S. Lei and S. Zhang, “Digital sinusoidal fringe pattern generation: defocusing binary patterns vs focusing sinusoidal patterns,” Opt. Laser Technol. |

16. | J. A. N. Buytaert and J. J. J. Dirckx, “Phase-shifting moiré topography using optical demodulation on liquid crystal matrices,” Opt. Lasers Eng. |

17. | C. S. McCamy, H. Marcus, and J. G. Davidson, “A color-rendition chart,” J. Appl. Photogr. Eng. |

18. | N. E. Huang, Z. Shen, S. R. Long, M. C. Wu, H. H. Shih, Q. Zheng, N.-C. Yen, C. C. Tung, and H. H. Liu, “The empirical mode decomposition and the Hilbert spectrum for nonlinear and nonstationary time series analysis,” Proc. R. Soc. Lond. A |

19. | P. Flandrin, http://perso.ens-lyon.fr/patrick.flandrin/emd.html. |

20. | S. Equis, “Phase extraction of non-stationary signals produced in dynamic interferometry involving speckle waves,” EPFL thesis n° |

21. | S. Equis and P. Jacquot, “Phase extraction in dynamic speckle interferometry by empirical mode decomposition and Hilbert transform,” Strain |

22. | S. Equis and P. Jacquot, “The empirical mode decomposition: a must-have tool in speckle interferometry?” Opt. Express |

23. | H. A. Aebischer and S. Waldner, “Simple and effective method for filtering speckle interferometric phase fringe patterns,” Opt. Commun. |

**OCIS Codes**

(120.0280) Instrumentation, measurement, and metrology : Remote sensing and sensors

(120.2650) Instrumentation, measurement, and metrology : Fringe analysis

(120.5050) Instrumentation, measurement, and metrology : Phase measurement

(070.2025) Fourier optics and signal processing : Discrete optical signal processing

**ToC Category:**

Instrumentation, Measurement, and Metrology

**History**

Original Manuscript: November 29, 2010

Revised Manuscript: December 30, 2010

Manuscript Accepted: December 30, 2010

Published: January 11, 2011

**Citation**

Sébastien Equis, Raik Schnabel, and Pierre Jacquot, "Snap-shot profilometry with the Empirical Mode Decomposition and a 3-layer color sensor," Opt. Express **19**, 1284-1290 (2011)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-19-2-1284

Sort: Year | Journal | Reset

### References

- F. Chen, G. M. Brown, and M. Song, “Overview of three-dimensional shape measurement using optical methods,” Opt. Eng. 39(1), 10–22 (2000). [CrossRef]
- E. Stoykova, A. A. Alatan, P. Benzie, N. Grammalidis, S. Malassiotis, J. Ostermann, S. Piekh, V. Sainov, C. Theobalt, T. Thevar, and X. Zabulis, “3-D time varying scene capture technologies – a survey,” IEEE Trans. Circ. Syst. Video Tech. 17(11), 1568–1586 (2007). [CrossRef]
- S. Zhang, “Recent progresses on real-time 3D shape measurement using digital fringe projection techniques,” Opt. Laser Technol. 48, 149–158 (2010).
- P. S. Huang, C. Zhang, and F.-P. Chiang, “High-speed 3-D shape measurement based on digital fringe projection,” Opt. Eng. 42(1), 163–168 (2003). [CrossRef]
- K. Creath, Interferogram Analysis (Institute of Physics Publishing, Bristol, UK, 1993), Chap. 4.
- P. Groot, “Derivation of algorithms for phase-shifting interferometry using the concept of a data-sampling window,” Appl. Opt. 34(22), 4723–4730 (1995). [CrossRef] [PubMed]
- D. W. Phillion, “General methods for generating phase-shifting interferometry algorithms,” Appl. Opt. 36(31), 8098–8115 (1997). [CrossRef]
- Y. Surrel, “Design of algorithms for phase measurements by the use of phase stepping,” Appl. Opt. 35(1), 51–60 (1996). [CrossRef] [PubMed]
- J. Novak, P. Novak, and A. Miks, “Multi-step phase-shifting algorithms insensitive to linear phase shift errors,” Opt. Commun. 281(21), 5302–5309 (2008). [CrossRef]
- X. Colonna de Lega and P. Jacquot, “Deformation measurement with object-induced dynamic phase shifting,” Appl. Opt. 35(25), 5115–5121 (1996). [CrossRef] [PubMed]
- C. Wust and D. W. Capson, “Surface profile measurement using color fringe projection,” Mach. Vis. Appl. 4(3), 193–203 (1991). [CrossRef]
- http://www.foveon.com .
- P. Tankam, P. Picart, D. Mounier, J. M. Desse, and J. C. Li, “Method of digital holographic recording and reconstruction using a stacked color image sensor,” Appl. Opt. 49(3), 320–328 (2010). [CrossRef] [PubMed]
- http://www.alt-vision.com/ .
- S. Lei and S. Zhang, “Digital sinusoidal fringe pattern generation: defocusing binary patterns vs focusing sinusoidal patterns,” Opt. Laser Technol. 48, 561–569 (2010).
- J. A. N. Buytaert and J. J. J. Dirckx, “Phase-shifting moiré topography using optical demodulation on liquid crystal matrices,” Opt. Lasers Eng. 48(2), 172–181 (2010). [CrossRef]
- C. S. McCamy, H. Marcus, and J. G. Davidson, “A color-rendition chart,” J. Appl. Photogr. Eng. 2(3), 95–99 (1976).
- N. E. Huang, Z. Shen, S. R. Long, M. C. Wu, H. H. Shih, Q. Zheng, N.-C. Yen, C. C. Tung, and H. H. Liu, “The empirical mode decomposition and the Hilbert spectrum for nonlinear and nonstationary time series analysis,” Proc. R. Soc. Lond. A 454(1971), 903–995 (1998). [CrossRef]
- P. Flandrin, http://perso.ens-lyon.fr/patrick.flandrin/emd.html .
- S. Equis, “Phase extraction of non-stationary signals produced in dynamic interferometry involving speckle waves,” EPFL thesis n° 4514, Lausanne, (2009). http://biblion.epfl.ch/EPFL/theses/2009/4514/EPFL_TH4514.pdf .
- S. Equis and P. Jacquot, “Phase extraction in dynamic speckle interferometry by empirical mode decomposition and Hilbert transform,” Strain 46(6), 550–558 (2010). [CrossRef]
- S. Equis and P. Jacquot, “The empirical mode decomposition: a must-have tool in speckle interferometry?” Opt. Express 17(2), 611–623 (2009). [CrossRef] [PubMed]
- H. A. Aebischer and S. Waldner, “Simple and effective method for filtering speckle interferometric phase fringe patterns,” Opt. Commun. 162(4-6), 205–210 (1999). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.