## Three-dimensional polarimetric computational integral imaging |

Optics Express, Vol. 20, Issue 14, pp. 15481-15488 (2012)

http://dx.doi.org/10.1364/OE.20.015481

Acrobat PDF (1170 KB)

### Abstract

In this paper, we propose a novel 3D polarimetric computational integral imaging system by using polarization diversity of objects under natural illumination conditions. In the system, the measured Stokes polarization parameters are utilized to generate degree of polarization images of a 3D scene. Based on degree of polarization images and original 2D images, we utilize a modified computational reconstruction method to perform 3D polarimetric image reconstruction. The system may be used to detect or classify objects with distinct polarization signatures in 3D space. Experimental results also show the proposed system may mitigate the effect of occlusion in 3D reconstruction.

© 2012 OSA

## 1. Introduction

1. H. S. Chen and C. Rao, “Polarization of light on reflection by some natural surfaces,” J. Phys. D **1**(9), 1191–1200 (1968). [CrossRef]

12. S. H. Hong, J. S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express **12**(3), 483–491 (2004). [CrossRef] [PubMed]

21. O. Matoba and B. Javidi, “Three-dimensional polarimetric integral imaging,” Opt. Lett. **29**(20), 2375–2377 (2004). [CrossRef] [PubMed]

24. P. Miché, A. Bensrhair, and D. Lebrun, “Passive 3-D shape recovery of unknown objects using cooperative polarimetric and radiometric stereo vision processes,” Opt. Eng. **44**(2), 027005 (2005). [CrossRef]

21. O. Matoba and B. Javidi, “Three-dimensional polarimetric integral imaging,” Opt. Lett. **29**(20), 2375–2377 (2004). [CrossRef] [PubMed]

22. B. Javidi, S. H. Hong, and O. Matoba, “Multidimensional optical sensor and imaging system,” Appl. Opt. **45**(13), 2986–2994 (2006). [CrossRef] [PubMed]

23. F. A. Sadjadi, “Passive three-dimensional imaging using polarimetric diversity,” Opt. Lett. **32**(3), 229–231 (2007). [CrossRef] [PubMed]

24. P. Miché, A. Bensrhair, and D. Lebrun, “Passive 3-D shape recovery of unknown objects using cooperative polarimetric and radiometric stereo vision processes,” Opt. Eng. **44**(2), 027005 (2005). [CrossRef]

25. K. E. Torrance and E. M. Sparrow, “Theory for off-specular reflection from roughened surfaces,” J. Opt. Soc. Am. A **57**(9), 1105–1112 (1967). [CrossRef]

26. R. B. Reid, M. E. Oxley, M. T. Eismann, and M. E. Goda, “Quantifying surface normal estimation,” Proc. SPIE **6240**, 624001, 624001-11 (2006). [CrossRef]

## 2. Degree of polarization images

*S*is the Stokes parameter of interest (

_{i}*i*= [0, 1, 2, 3]),

*E*and

_{x}*E*are the maximum amplitudes of the

_{y}*x*and

*y*components of the electric field, and

*δ*is the phase difference between the orthogonal components of the electric field. Angular brackets in (1) denote a temporal average. Note that a general beam is composed of natural (unpolarized) and completely polarized light. The parameter

*S*represents the total irradiance of the electric field while

_{0}*S*describes the relationship between the irradiance of linearly horizontally polarized and linearly vertically polarized components in the light beam. Similarly,

_{1}*S*describes the relationship between the linear + 45þ polarized component and the linear −45þ polarized component, and

_{2}*S*characterizes the part of circular polarization in the field. For completely unpolarized light, only

_{3}*S*remains. If the light is completely linearly polarized,

_{0}*S*is zero. Moreover, for completely polarized light,

_{3}29. B. Schaefer, E. Collett, R. Smyth, D. Barrett, and B. Fraher, “Measuring the stokes polarization parameters,” Am. J. Phys. **75**(2), 163–168 (2007). [CrossRef]

*θ*= 0°, 90°, 45° and 135° with respect to the

*x*-axis of the Cartesian coordinate system (intensities

*I*

_{1},

*I*

_{2},

*I*

_{3}and

*I*

_{4}, respectively). The final measurement is made by adding the QWP into the optical train before the LP, with its fast axis lying on the

*x*-axis of the reference system (intensity

*I*

_{5}). For the final measurement, the angle of the LP is set to

*θ*= 45°.

*S*.

_{3}26. R. B. Reid, M. E. Oxley, M. T. Eismann, and M. E. Goda, “Quantifying surface normal estimation,” Proc. SPIE **6240**, 624001, 624001-11 (2006). [CrossRef]

*S*.

_{3}## 3. 3D polarimetric computational integral imaging system

### 3.1 Pickup of polarimetric images

15. A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE **94**, 591–608 (2006). [CrossRef]

30. J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. **27**(13), 1144–1146 (2002). [CrossRef] [PubMed]

### 3.2 Polarimetric information applied to computational image reconstruction

12. S. H. Hong, J. S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express **12**(3), 483–491 (2004). [CrossRef] [PubMed]

*N*, the image at the reconstruction plane,

*R*(

*x*,

*y*;

*z*), at a distance

*z*from the imaging system, is expressed as

*EI*represents the

_{k}*k*-th elemental image,

*M*is the magnification factor (

*M*=

*z*/

*g*,

*g*being the distance between the pickup plane and the image plane),

*r*describes the number of pixels per unit distance for the

^{k}*k*-th sensor,

*k*-th sensor,

*k*-th DoP image,

*I*(∙) is an indicator function,

*p*is a given threshold of degree of polarization (0 ≤

*p*≤ 1), and

*N*is a given number (0 <

_{t}*N*<

_{t}*N*).

*p*are used to perform the 3D image reconstruction.

*T*is the total number of the pixels whose DoP are larger than the given threshold

*p*. In this way, we can implement polarimetric image reconstruction in 3D space. The condition in Eq. (4a),

*T*>

*N*, is used to reduce reconstruction errors caused by some artificial DoP information.

_{t}*N*can be set as

_{t}*N*/4 or

*N*/5 based on the particular situation.

## 4. Experimental results

**®**5D series. According to the quantum efficiencies of the three channels of this camera (see files from http://astrosurf.com/buil/50d/test.htm), we decided to use the G channel and a QWP designed for 543 nm from Melles Griot

**®**. The spectral band tolerance width of this QWP is around 80 nm which matches the efficiency width of the G channel of our sensor. Moreover, the transmittance of the selected QWP is larger than 99.75%. We can ignore the absorption of our QWP when using Eq. (2) to calculate DoP information.

*green*objects to compose the scene because we use the G channel of the image sensor. DoLP, DoCP and DoP images of the corresponding images in Fig. 4 are shown in Fig. 5 with the given color mapping. It can easily be seen that light coming from most of the surfaces of the cars is highly polarized in contrast to the ones of the occlusion and the background. Moreover, most of the polarization generated after reflection of the natural light is linear. From Fig. 5, we can also see a relatively high value of the polarization degrees around the edges of the occlusion, objects and background. In fact, these values are likely artifacts since the calculation of DoP for the image edges may have some errors because the images captured to measure Stokes parameters may have small shift bias when we rotate the LP to different angles (see Eq. (3)). If the shift distances among the images are large (more than 2 or 3 pixels), some methods should be used to obtain the corresponding shifted pixels and correct the images in order to reduce the measurement errors. In our experiment, we calculated the shift distances by using local correlation, which are approximately 1 pixel.

*p*for the DoP is set to 0.2 and 0.4, and

*N*is fixed to

_{t}*N*/5. In Fig. 6(b)-6(c), only the cars show up but the occlusion and the background do not appear on thereconstruction images. This occurs because in our polarimetric integral imaging system only the objects emitting light with DoP larger than

*p*are reconstructed in 3D space. Experimental results also show that occlusion can be mitigated in 3D image reconstruction when occlusion has a diffuse surface compared to the objects of interest. Moreover, the 3D reconstruction method may benefit the tasks of specific object detection or classification. However, in practical applications, the direction of the light source may need to be considered as the DoP images may vary with the orientation of illumination.

## 5. Conclusion

## Acknowledgments

## References and links

1. | H. S. Chen and C. Rao, “Polarization of light on reflection by some natural surfaces,” J. Phys. D |

2. | T. H. Waterman, “Polarization sensitivity,” Handbook of sensory physiology |

3. | L. B. Wolff, “Polarization-based material classification from specular reflection,” IEEE Trans. Pattern Anal. Mach. Intell. |

4. | L. B. Wolff, “Polarization vision: a new sensory approach to image understanding,” Image Vision Comput. |

5. | S. Daly, “Polarimetric imaging,” Rochester Institute of Technology, 2002, (technical report). |

6. | M. I. Mishchenko, Y. S. Yatskiv, V. K. Rosenbush, and G. Videen, |

7. | G. Lippmann, “La photographie integrale,” CR Acad. Sci. |

8. | H. E. Ives, “Optical properties of a Lippman lenticulated sheet,” J. Opt. Soc. Am. A |

9. | C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. |

10. | T. Okoshi, “Three-dimensional displays,” Proc. IEEE |

11. | H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett. |

12. | S. H. Hong, J. S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express |

13. | M. Martinez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express |

14. | F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE |

15. | A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE |

16. | M. DaneshPanah and B. Javidi, “Profilometry and optical slicing by passive three-dimensional imaging,” Opt. Lett. |

17. | R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE |

18. | B. Javidi, F. Okano, and J. Y. Son, |

19. | M. Pollefeys, R. Koch, M. Vergauwen, A. A. Deknuydt, and L. J. Van Gool, “Three-dimensional scene reconstruction from images,” Proc. SPIE |

20. | L. Guan, J. S. Franco, E. Boyer, and M. Pollefeys, “Probabilistic 3D occupancy flow with latent silhouette cues,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1379–1386 (2010). |

21. | O. Matoba and B. Javidi, “Three-dimensional polarimetric integral imaging,” Opt. Lett. |

22. | B. Javidi, S. H. Hong, and O. Matoba, “Multidimensional optical sensor and imaging system,” Appl. Opt. |

23. | F. A. Sadjadi, “Passive three-dimensional imaging using polarimetric diversity,” Opt. Lett. |

24. | P. Miché, A. Bensrhair, and D. Lebrun, “Passive 3-D shape recovery of unknown objects using cooperative polarimetric and radiometric stereo vision processes,” Opt. Eng. |

25. | K. E. Torrance and E. M. Sparrow, “Theory for off-specular reflection from roughened surfaces,” J. Opt. Soc. Am. A |

26. | R. B. Reid, M. E. Oxley, M. T. Eismann, and M. E. Goda, “Quantifying surface normal estimation,” Proc. SPIE |

27. | E. Collett, |

28. | E. Wolf, |

29. | B. Schaefer, E. Collett, R. Smyth, D. Barrett, and B. Fraher, “Measuring the stokes polarization parameters,” Am. J. Phys. |

30. | J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett. |

**OCIS Codes**

(100.6890) Image processing : Three-dimensional image processing

(110.6880) Imaging systems : Three-dimensional image acquisition

(120.5410) Instrumentation, measurement, and metrology : Polarimetry

(150.6910) Machine vision : Three-dimensional sensing

**ToC Category:**

Imaging Systems

**History**

Original Manuscript: March 23, 2012

Revised Manuscript: May 25, 2012

Manuscript Accepted: May 28, 2012

Published: June 25, 2012

**Citation**

Xiao Xiao, Bahram Javidi, Genaro Saavedra, Michael Eismann, and Manuel Martinez-Corral, "Three-dimensional polarimetric computational integral imaging," Opt. Express **20**, 15481-15488 (2012)

http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-20-14-15481

Sort: Year | Journal | Reset

### References

- H. S. Chen and C. Rao, “Polarization of light on reflection by some natural surfaces,” J. Phys. D1(9), 1191–1200 (1968). [CrossRef]
- T. H. Waterman, “Polarization sensitivity,” Handbook of sensory physiology 7, 281–469 (1981).
- L. B. Wolff, “Polarization-based material classification from specular reflection,” IEEE Trans. Pattern Anal. Mach. Intell.12(11), 1059–1071 (1990). [CrossRef]
- L. B. Wolff, “Polarization vision: a new sensory approach to image understanding,” Image Vision Comput.15, 81–93 (1997).
- S. Daly, “Polarimetric imaging,” Rochester Institute of Technology, 2002, (technical report).
- M. I. Mishchenko, Y. S. Yatskiv, V. K. Rosenbush, and G. Videen, Polarimetric Detection, Characterization, and Remote Sensing (Springer, 2011).
- G. Lippmann, “La photographie integrale,” CR Acad. Sci.146, 446–451 (1908).
- H. E. Ives, “Optical properties of a Lippman lenticulated sheet,” J. Opt. Soc. Am. A21(3), 171–176 (1931). [CrossRef]
- C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am.58(1), 71–74 (1968). [CrossRef]
- T. Okoshi, “Three-dimensional displays,” Proc. IEEE68(5), 548–564 (1980). [CrossRef]
- H. Arimoto and B. Javidi, “Integral three-dimensional imaging with digital reconstruction,” Opt. Lett.26(3), 157–159 (2001). [CrossRef] [PubMed]
- S. H. Hong, J. S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express12(3), 483–491 (2004). [CrossRef] [PubMed]
- M. Martinez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Formation of real, orthoscopic integral images by smart pixel mapping,” Opt. Express13(23), 9175–9180 (2005). [CrossRef] [PubMed]
- F. Okano, J. Arai, K. Mitani, and M. Okui, “Real-time integral imaging based on extremely high resolution video system,” Proc. IEEE94(3), 490–501 (2006). [CrossRef]
- A. Stern and B. Javidi, “3D image sensing, visualization, and processing using integral imaging,” Proc. IEEE94, 591–608 (2006). [CrossRef]
- M. DaneshPanah and B. Javidi, “Profilometry and optical slicing by passive three-dimensional imaging,” Opt. Lett.34(7), 1105–1107 (2009). [CrossRef] [PubMed]
- R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Progress in 3-D multiperspective display by integral imaging,” Proc. IEEE97(6), 1067–1077 (2009). [CrossRef]
- B. Javidi, F. Okano, and J. Y. Son, Three-dimensional Imaging, Visualization, and Display (Springer, 2009).
- M. Pollefeys, R. Koch, M. Vergauwen, A. A. Deknuydt, and L. J. Van Gool, “Three-dimensional scene reconstruction from images,” Proc. SPIE3958, 215–226 (2000). [CrossRef]
- L. Guan, J. S. Franco, E. Boyer, and M. Pollefeys, “Probabilistic 3D occupancy flow with latent silhouette cues,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1379–1386 (2010).
- O. Matoba and B. Javidi, “Three-dimensional polarimetric integral imaging,” Opt. Lett.29(20), 2375–2377 (2004). [CrossRef] [PubMed]
- B. Javidi, S. H. Hong, and O. Matoba, “Multidimensional optical sensor and imaging system,” Appl. Opt.45(13), 2986–2994 (2006). [CrossRef] [PubMed]
- F. A. Sadjadi, “Passive three-dimensional imaging using polarimetric diversity,” Opt. Lett.32(3), 229–231 (2007). [CrossRef] [PubMed]
- P. Miché, A. Bensrhair, and D. Lebrun, “Passive 3-D shape recovery of unknown objects using cooperative polarimetric and radiometric stereo vision processes,” Opt. Eng.44(2), 027005 (2005). [CrossRef]
- K. E. Torrance and E. M. Sparrow, “Theory for off-specular reflection from roughened surfaces,” J. Opt. Soc. Am. A57(9), 1105–1112 (1967). [CrossRef]
- R. B. Reid, M. E. Oxley, M. T. Eismann, and M. E. Goda, “Quantifying surface normal estimation,” Proc. SPIE6240, 624001, 624001-11 (2006). [CrossRef]
- E. Collett, Polarized Light: Fundamentals and Applications (Marcel Dekker, 1993).
- E. Wolf, Introduction to the Theory of Coherence and Polarization of Light (Cambridge University Press, 2007).
- B. Schaefer, E. Collett, R. Smyth, D. Barrett, and B. Fraher, “Measuring the stokes polarization parameters,” Am. J. Phys.75(2), 163–168 (2007). [CrossRef]
- J. S. Jang and B. Javidi, “Three-dimensional synthetic aperture integral imaging,” Opt. Lett.27(13), 1144–1146 (2002). [CrossRef] [PubMed]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.