OSA's Digital Library

Journal of the Optical Society of America A

Journal of the Optical Society of America A


  • Editor: Franco Gori
  • Vol. 28, Iss. 2 — Feb. 1, 2011
  • pp: 136–146

Variational estimation of inhomogeneous specular reflectance and illumination from a single view

Kenji Hara and Ko Nishino  »View Author Affiliations

JOSA A, Vol. 28, Issue 2, pp. 136-146 (2011)

View Full Text Article

Enhanced HTML    Acrobat PDF (818 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



Estimating the illumination and the reflectance properties of an object surface from a few images is an important but challenging problem. The problem becomes even more challenging if we wish to deal with real-world objects that naturally have spatially inhomogeneous reflectance. In this paper, we derive a novel method for estimating the spatially varying specular reflectance properties of a surface of known geometry as well as the illumination distribution of a scene from a specular-only image, for instance, recovered from two images captured with a polarizer to separate reflection components. Unlike previous work, we do not assume the illumination to be a single point light source. We model specular reflection with a spherical statistical distribution and encode its spatial variation with a radial basis function (RBF) network of their parameter values, which allows us to formulate the simultaneous estimation of spatially varying specular reflectance and illumination as a constrained optimization based on the I-divergence measure. To solve it, we derive a variational algorithm based on the expectation maximization principle. At the same time, we estimate optimal encoding of the specular reflectance properties by learning the number, centers, and widths of the RBF hidden units. We demonstrate the effectiveness of the method on images of synthetic and real-world objects.

© 2011 Optical Society of America

OCIS Codes
(100.2960) Image processing : Image analysis
(100.3190) Image processing : Inverse problems
(150.2950) Machine vision : Illumination

ToC Category:
Image Processing

Original Manuscript: July 16, 2010
Revised Manuscript: November 24, 2010
Manuscript Accepted: November 25, 2010
Published: January 17, 2011

Kenji Hara and Ko Nishino, "Variational estimation of inhomogeneous specular reflectance and illumination from a single view," J. Opt. Soc. Am. A 28, 136-146 (2011)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. G. Kay and T. Caelli, “Inverting an illumination model from range and intensity maps,” CVGIP, Image Underst. 59, 183–201(1994). [CrossRef]
  2. Y. Sato, M. Wheeler, and K. Ikeuchi, “Object shape and reflectance modeling from observation,” in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (Association for Computing Machinery, 1997), pp. 379–387.
  3. P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, “Acquiring the reflectance field of a human face,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (Association for Computing Machinery, 2000), pp. 145–156.
  4. S. Marschner and D. Greenberg, “Inverse lighting for photography,” In Proceedings of IS&T/SID Fifth Color Imaging Conference (The Society for Imaging Science and Technology, 1997), pp. 262–265.
  5. K. Hara, K. Nishino, and K. Ikeuchi, “Mixture of spherical distributions for single-view relighting,” IEEE Trans. Patt. Anal. Mach. Intell. 30, 25–35 (2008). [CrossRef]
  6. K. Nishino, K. Ikeuchi, and Z. Zhang, “Re-rendering from a sparse set of images,” Tech. Rep. DU-CS-05-12 (Drexel University, 2005).
  7. R. Ramamoorthi and P. Hanrahan, “A signal processing framework for inverse rendering,” in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (Association for Computing Machinery, 2001), pp. 117–128.
  8. H. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H. Seidel, “Image-based reconstruction of spatially varying materials,” in Proceedings of the 12th Eurographics Workshop on Rendering Techniques, S.J.Gortler and K.Myszkowski, eds. (Springer, 2001), pp. 104–115.
  9. T. Zickler, R. Ramamoorthi, S. Enrique, and P. Belhumeur, “Reflectance sharing: predicting appearance from a sparse set of images of a known shape,” IEEE Trans. Patt. Anal. Mach. Intell. 28, 1287–1302 (2006). [CrossRef]
  10. D. Goldman, B. Curless, A. Hertzmann, and S. Seitz, “Shape and spatially-varying BRDFs from photometric stereo,” IEEE Trans. Patt. Anal. Mach. Intell. 32, 1060–1071 (2010). [CrossRef]
  11. I. Csiszár, “Why least squares and maximum entropy? An axiomatic approach to inverse problems,” Ann. Stat. 19, 2032–2066(1991). [CrossRef]
  12. A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc. Ser. B. Methodol. 39, 1–38 (1977). [CrossRef]
  13. S. Nayar, K. Ikeuchi, and T. Kanade, “Determining shape and reflectance of Lambertian, specular, and hybrid surfaces using extended sources,” in Proceedings of the International Workshop on Industrial Applications of Machine Intelligence and Vision (IEEE, 1989), pp. 169–175. [CrossRef]
  14. Y. Sato and K. Ikeuchi, “Temporal-color space analysis of reflection,” J. Opt. Soc. Am. A 11, 2990–3002 (1994). [CrossRef]
  15. A. L. Yuille, D. Snow, R. Epstein, and P. N. Belhumeur, “Determining generative models of objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vis. 35, 203–222 (1999). [CrossRef]
  16. W. Y. Zhao and R. Chellappa, “Symmetric shape-from-shading using self-ratio image,” Int. J. Comput. Vis. 45, 55–75(2001). [CrossRef]
  17. Q. Zheng and R. Chellappa, “Estimation of illuminant direction, albedo and shape from shading,” IEEE Trans. Patt. Anal. Mach. Intell. 13, 680–702 (1991). [CrossRef]
  18. W. A. P. Smith and E. R. Hancock, “Recovering facial shape using a statistical model of surface normal direction,” IEEE Trans. Patt. Anal. Mach. Intell. 28, 1914–1930 (2006). [CrossRef]
  19. N. Birkbeck, D. Cobzas, P. F. Sturm, and M. Jagersand, “Variational shape and reflectance estimation under changing light and viewpoints,” in Proceedings of the 9th European Conference on Computer Vision (Springer, 2006), pp. 536–549.
  20. N. Alldrin, T. Zickler, and D. Kriegman, “Photometric stereo with non-parametric and spatially-varying reflectance,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8.
  21. S. Biswas, G. Aggarwal, and R. Chellappa, “Robust estimation of albedo for illumination-invariant matching and shape recovery,” IEEE Trans. Patt. Anal. Mach. Intell. 31, 884–899(2009). [CrossRef]
  22. K. Ikeuchi and K. Sato, “Determining reflectance properties of an object using range and brightness images,” IEEE Trans. Patt. Anal. Mach. Intell. 13, 1139–1153 (1991). [CrossRef]
  23. I. Sato, Y. Sato, and K. Ikeuchi, “Illumination from shadows,” IEEE Trans. Patt. Anal. Mach. Intell. 25, 290–300 (2003). [CrossRef]
  24. S. Tominaga and N. Tanaka, “Estimating reflection parameters from a single color image,” IEEE Comput. Graph. Appl. 20, 58–66 (2000). [CrossRef]
  25. T. Yu, H. Wang, N.Ahuja, and W.-C. Chen, “Sparse lumigraph relighting by illumination and reflectance estimation from multi-view images,” in Rendering Techniques 2006: Eurographics Symposium on Rendering, T.Akenine-Mo¨ller and W.Heidrich, eds. (Eurographics Association, 2006), pp. 41–50.
  26. K. E.Torrance and E. M.Sparrow, “Theory for off-specular reflection from roughened surfaces,” J. Opt. Soc. Am. 57, 1105–1112 (1967). [CrossRef]
  27. R. Fisher, “Dispersion on a sphere,” Proc. R. Soc. Lond., Ser. A. 217, 295–305 (1953). [CrossRef]
  28. K. Mardia and P. Jupp, Directional Statistics (Wiley, 2000).
  29. F. Solomon and K. Ikeuchi, “Extracting the shape and roughness of specular lobe objects using four light photometric stereo,” IEEE Trans. Patt. Anal. Mach. Intell. 18, 449–454 (1996). [CrossRef]
  30. H. Kameoka, T. Nishimoto, and S. Sagayama, “A multipitch analyzer based on harmonic temporal structured clustering,” IEEE Trans. Audio Speech Lang. Process. 15, 982–994(2007). [CrossRef]
  31. N. Ueda and R. Nakano, “Deterministic annealing EM algorithm,” Neural Netw. 11, 271–282 (1998). [CrossRef]
  32. M. Lavielle and E. Moulines, “A simulated annealing version of the EM algorithm for non-Gaussian deconvolution,” Stat. Comput. 7, 229–236 (1997). [CrossRef]
  33. L. Yingwei, N. Sundararajan, and P. Saratchandran, “A sequential learning scheme for function approximation using minimal radial basis function neural networks,” Neural Comput. 9,461–478 (1997). [CrossRef]
  34. S. Nayar, X. Fang, and T. Boult, “Removal of specularities using color and polarization,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1993), pp. 583–590. [CrossRef]
  35. L. Wolff and T. Boult, “Constraining object features using a polarization reflectance model,” IEEE Trans. Patt. Anal. Mach. Intell. 13, 635–657 (1991). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited