## Variational estimation of inhomogeneous specular reflectance and illumination from a single view |

JOSA A, Vol. 28, Issue 2, pp. 136-146 (2011)

http://dx.doi.org/10.1364/JOSAA.28.000136

Enhanced HTML Acrobat PDF (818 KB)

### Abstract

Estimating the illumination and the reflectance properties of an object surface from a few images is an important but challenging problem. The problem becomes even more challenging if we wish to deal with real-world objects that naturally have spatially inhomogeneous reflectance. In this paper, we derive a novel method for estimating the spatially varying specular reflectance properties of a surface of known geometry as well as the illumination distribution of a scene from a specular-only image, for instance, recovered from two images captured with a polarizer to separate reflection components. Unlike previous work, we do not assume the illumination to be a single point light source. We model specular reflection with a spherical statistical distribution and encode its spatial variation with a radial basis function (RBF) network of their parameter values, which allows us to formulate the simultaneous estimation of spatially varying specular reflectance and illumination as a constrained optimization based on the *I*-divergence measure. To solve it, we derive a variational algorithm based on the expectation maximization principle. At the same time, we estimate optimal encoding of the specular reflectance properties by learning the number, centers, and widths of the RBF hidden units. We demonstrate the effectiveness of the method on images of synthetic and real-world objects.

© 2011 Optical Society of America

**OCIS Codes**

(100.2960) Image processing : Image analysis

(100.3190) Image processing : Inverse problems

(150.2950) Machine vision : Illumination

**ToC Category:**

Image Processing

**History**

Original Manuscript: July 16, 2010

Revised Manuscript: November 24, 2010

Manuscript Accepted: November 25, 2010

Published: January 17, 2011

**Citation**

Kenji Hara and Ko Nishino, "Variational estimation of inhomogeneous specular reflectance and illumination from a single view," J. Opt. Soc. Am. A **28**, 136-146 (2011)

http://www.opticsinfobase.org/josaa/abstract.cfm?URI=josaa-28-2-136

Sort: Year | Journal | Reset

### References

- G. Kay and T. Caelli, “Inverting an illumination model from range and intensity maps,” CVGIP, Image Underst. 59, 183–201(1994). [CrossRef]
- Y. Sato, M. Wheeler, and K. Ikeuchi, “Object shape and reflectance modeling from observation,” in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (Association for Computing Machinery, 1997), pp. 379–387.
- P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, “Acquiring the reflectance field of a human face,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (Association for Computing Machinery, 2000), pp. 145–156.
- S. Marschner and D. Greenberg, “Inverse lighting for photography,” In Proceedings of IS&T/SID Fifth Color Imaging Conference (The Society for Imaging Science and Technology, 1997), pp. 262–265.
- K. Hara, K. Nishino, and K. Ikeuchi, “Mixture of spherical distributions for single-view relighting,” IEEE Trans. Patt. Anal. Mach. Intell. 30, 25–35 (2008). [CrossRef]
- K. Nishino, K. Ikeuchi, and Z. Zhang, “Re-rendering from a sparse set of images,” Tech. Rep. DU-CS-05-12 (Drexel University, 2005).
- R. Ramamoorthi and P. Hanrahan, “A signal processing framework for inverse rendering,” in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (Association for Computing Machinery, 2001), pp. 117–128.
- H. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H. Seidel, “Image-based reconstruction of spatially varying materials,” in Proceedings of the 12th Eurographics Workshop on Rendering Techniques, S.J.Gortler and K.Myszkowski, eds. (Springer, 2001), pp. 104–115.
- T. Zickler, R. Ramamoorthi, S. Enrique, and P. Belhumeur, “Reflectance sharing: predicting appearance from a sparse set of images of a known shape,” IEEE Trans. Patt. Anal. Mach. Intell. 28, 1287–1302 (2006). [CrossRef]
- D. Goldman, B. Curless, A. Hertzmann, and S. Seitz, “Shape and spatially-varying BRDFs from photometric stereo,” IEEE Trans. Patt. Anal. Mach. Intell. 32, 1060–1071 (2010). [CrossRef]
- I. Csiszár, “Why least squares and maximum entropy? An axiomatic approach to inverse problems,” Ann. Stat. 19, 2032–2066(1991). [CrossRef]
- A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc. Ser. B. Methodol. 39, 1–38 (1977). [CrossRef]
- S. Nayar, K. Ikeuchi, and T. Kanade, “Determining shape and reflectance of Lambertian, specular, and hybrid surfaces using extended sources,” in Proceedings of the International Workshop on Industrial Applications of Machine Intelligence and Vision (IEEE, 1989), pp. 169–175. [CrossRef]
- Y. Sato and K. Ikeuchi, “Temporal-color space analysis of reflection,” J. Opt. Soc. Am. A 11, 2990–3002 (1994). [CrossRef]
- A. L. Yuille, D. Snow, R. Epstein, and P. N. Belhumeur, “Determining generative models of objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vis. 35, 203–222 (1999). [CrossRef]
- W. Y. Zhao and R. Chellappa, “Symmetric shape-from-shading using self-ratio image,” Int. J. Comput. Vis. 45, 55–75(2001). [CrossRef]
- Q. Zheng and R. Chellappa, “Estimation of illuminant direction, albedo and shape from shading,” IEEE Trans. Patt. Anal. Mach. Intell. 13, 680–702 (1991). [CrossRef]
- W. A. P. Smith and E. R. Hancock, “Recovering facial shape using a statistical model of surface normal direction,” IEEE Trans. Patt. Anal. Mach. Intell. 28, 1914–1930 (2006). [CrossRef]
- N. Birkbeck, D. Cobzas, P. F. Sturm, and M. Jagersand, “Variational shape and reflectance estimation under changing light and viewpoints,” in Proceedings of the 9th European Conference on Computer Vision (Springer, 2006), pp. 536–549.
- N. Alldrin, T. Zickler, and D. Kriegman, “Photometric stereo with non-parametric and spatially-varying reflectance,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008), pp. 1–8.
- S. Biswas, G. Aggarwal, and R. Chellappa, “Robust estimation of albedo for illumination-invariant matching and shape recovery,” IEEE Trans. Patt. Anal. Mach. Intell. 31, 884–899(2009). [CrossRef]
- K. Ikeuchi and K. Sato, “Determining reflectance properties of an object using range and brightness images,” IEEE Trans. Patt. Anal. Mach. Intell. 13, 1139–1153 (1991). [CrossRef]
- I. Sato, Y. Sato, and K. Ikeuchi, “Illumination from shadows,” IEEE Trans. Patt. Anal. Mach. Intell. 25, 290–300 (2003). [CrossRef]
- S. Tominaga and N. Tanaka, “Estimating reflection parameters from a single color image,” IEEE Comput. Graph. Appl. 20, 58–66 (2000). [CrossRef]
- T. Yu, H. Wang, N.Ahuja, and W.-C. Chen, “Sparse lumigraph relighting by illumination and reflectance estimation from multi-view images,” in Rendering Techniques 2006: Eurographics Symposium on Rendering, T.Akenine-Mo¨ller and W.Heidrich, eds. (Eurographics Association, 2006), pp. 41–50.
- K. E.Torrance and E. M.Sparrow, “Theory for off-specular reflection from roughened surfaces,” J. Opt. Soc. Am. 57, 1105–1112 (1967). [CrossRef]
- R. Fisher, “Dispersion on a sphere,” Proc. R. Soc. Lond., Ser. A. 217, 295–305 (1953). [CrossRef]
- K. Mardia and P. Jupp, Directional Statistics (Wiley, 2000).
- F. Solomon and K. Ikeuchi, “Extracting the shape and roughness of specular lobe objects using four light photometric stereo,” IEEE Trans. Patt. Anal. Mach. Intell. 18, 449–454 (1996). [CrossRef]
- H. Kameoka, T. Nishimoto, and S. Sagayama, “A multipitch analyzer based on harmonic temporal structured clustering,” IEEE Trans. Audio Speech Lang. Process. 15, 982–994(2007). [CrossRef]
- N. Ueda and R. Nakano, “Deterministic annealing EM algorithm,” Neural Netw. 11, 271–282 (1998). [CrossRef]
- M. Lavielle and E. Moulines, “A simulated annealing version of the EM algorithm for non-Gaussian deconvolution,” Stat. Comput. 7, 229–236 (1997). [CrossRef]
- L. Yingwei, N. Sundararajan, and P. Saratchandran, “A sequential learning scheme for function approximation using minimal radial basis function neural networks,” Neural Comput. 9,461–478 (1997). [CrossRef]
- S. Nayar, X. Fang, and T. Boult, “Removal of specularities using color and polarization,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 1993), pp. 583–590. [CrossRef]
- L. Wolff and T. Boult, “Constraining object features using a polarization reflectance model,” IEEE Trans. Patt. Anal. Mach. Intell. 13, 635–657 (1991). [CrossRef]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

« Previous Article | Next Article »

OSA is a member of CrossRef.