## Color constancy: a method for recovering surface spectral reflectance

JOSA A, Vol. 3, Issue 1, pp. 29-33 (1986)

http://dx.doi.org/10.1364/JOSAA.3.000029

Acrobat PDF (656 KB)

### Abstract

Human and machine visual sensing is enhanced when surface properties of objects in scenes, including color, can be reliably estimated despite changes in the ambient lighting conditions. We describe a computational method for estimating surface spectral reflectance when the spectral power distribution of the ambient light is not known.

© 1986 Optical Society of America

## INTRODUCTION

## PRELIMINARY DEFINITIONS

*E*(λ), which describes the energy per second at each wavelength, λ. The ambient light is reflected from surfaces and focused onto the sensor array. The proportion of light of wavelength λ reflected from an object toward location

*x*on the sensor array is determined by the surface spectral reflectance,

*S*

*(λ). The superscript*

^{x}*x*denotes the spatial position on the two-dimensional sensor array at which the object is imaged. [3]

3. In general the surface reflectance function may depend on the geometry of the scene, the angle of incidence of the light on the surface, and the angle between the surface and the line of sight. We are concerned here with the analysis of a single image drawn from a scene with fixed geometric relations among objects, light sources, and the visual sensor array. *S** ^{x}*(λ) refers to the proportion of light returned from the object toward the sensor array within that fixed geometrical framework.

*x*on the sensor array is described by the function

*E*(λ)

*S*

*(λ).*

^{x}*p*distinct classes of sensors at each location

*x*. In human vision, there are four photoreceptor classes (rods and cones), of which three (cones) are known to be active in daylight vision. We denote the relative wavelength sensitivity of the visual color sensors of the

*k*th class as

*R*

*(λ)*

_{k}*p*sensors at location

*x*records a sensor quantum catch

*p*sensor quantum catches at each location

*x*. The spectral reflectance at each location

*S*

*(λ) is assumed to be unknown.*

^{x}*ρ*

_{k}*, we show how to recover the surface spectral reflectances*

^{x}*S*

*(λ) over a range of possible ambient lights*

^{x}*E*(λ). Knowledge of

*S*

*(λ) permits us to compute color descriptors that are independent of the ambient light*

^{x}*E*(λ).

## PREVIOUS WORK

4. D. B. Judd, “Hue saturation and lightness of surface colors with chromatic illumination,” J. Opt. Soc. Am. **30**, 2 (1940); [CrossRef]

4. H. Helson, “Fundamental problems in color vision. I. The principle governing changes in hue saturation and lightness of non-selective samples in chromatic illumination,”J. Exp. Psychol. **23**, 439 (1938) [CrossRef] .

5. E. H. Land and J. J. McCann, “Lightness and retinex theory,”J. Opt. Soc. Am. **61**, 1 (1971); [CrossRef] [PubMed]

5. E. H. Land, “Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image,” Proc. Nat. Acad. Sci. U.S. **80**, 5163 (1983); [CrossRef]

5. E. H. Land, D. H. Hubel, M. Livingston, S. Perry, and M. Burns, “Colour-generating interactions across the corpus callosum,” Nature **303**, 616 (1983) [CrossRef] [PubMed] .

7. G. Buchsbaum, “A spatial processor model for object colour perception,”J. Franklin Inst. **310**, 1 (1980) [CrossRef] .

8. It is not possible to recover *E*(λ) better than to within a multiplicative constant given only the sensor quantum catches. If, for example, the intensity of the light is doubled to 2*E* but all reflectances are halved to ½*S** ^{x}*(λ), it is easy to verify that the sensor quantum catches in Eq. (1) are unchanged. When we speak of recovering the ambient light and surface reflectances, we mean recovery up to this unknown mutiplicative constant.

## MODELS OF LIGHTS AND SURFACES REFLECTANCES

*S*

*(λ) as suggested by Sällström, [1] Buchsbaum, [7]*

_{j}7. G. Buchsbaum, “A spatial processor model for object colour perception,”J. Franklin Inst. **310**, 1 (1980) [CrossRef] .

9. M. H. Brill, “A device performing illuminant-invariant assessment of chromatic relations,”J. Theor. Biol. **71**, 473 (1978) [CrossRef] [PubMed] .

*n*, is referred to as the number of degrees of freedom in the model. Knowledge of the weights

*σ*

_{j}*corresponding to a surface reflectance*

^{x}*S*

*(λ) described by the finite linear model amounts to complete knowledge of*

^{x}*S*

*(λ).*

^{x}*n*is large enough. What is surprising is that models with only a few basis reflectances provide excellent approximations to many naturally occurring spectral reflectances. Stiles

*et al*. [10]

10. W. S. Stiles, G. Wyszecki, and N. Ohta, “Counting metameric object-color stimuli using frequency-limited spectral reflectance functions,”J. Opt. Soc. Am. **67**, 779 (1977) [CrossRef] .

12. G. Buchsbaum and A. Gottschalk, “Chromaticity coordinates of frequency-limited functions,” J. Opt. Soc. Am. A **1**, 885 (1984) [CrossRef] [PubMed] .

15. E. Krinov, *Spectral Reflectance Properties of Natural Formations*, Technical translation TT-439 (National Research Council of Canada, Ottawa, 1947);

15. details of the fit of Cohen’s characteristic vectors to the Munsell surface reflectances are given in L. Maloney, “Computational approaches to color constancy,” Stanford Applied Psychology Lab. Tech. Rep. 1985-01 (Stanford University, Stanford, Calif., 1985).

*E*

*(λ). It is natural to inquire how well such a model captures the range of spectral variation of natural lights such as daylight. Judd*

_{i}*et al*. [16]

16. D. B. Judd, D. L. MacAdam, and G. Wyszecki, “Spectral distribution of typical daylight as a function of correlated color temperature,”J. Opt. Soc. Am. **54**, 1031 (1964) [CrossRef] .

*et al*. suggest that the number of parameters required to have an adequate linear model of the ambient light may often be small. Dixon and others have independently measured and analyzed spectral power distributions of daylight and drawn similar conclusions. [17]

17. E. R. Dixon, “Spectral distribution of Australian daylight,”J. Opt. Soc. Am. **68**, 437 (1978); [CrossRef]

17. G. T. Winch, M. C. Boshoff, C. J. Kok, and A. G. du Toit, “Spectroradiometric and calorimetric characteristics of daylight in the southern hemisphere: Pretoria, South Africa,”J. Opt. Soc. Am. **56**, 456 (1966); [CrossRef]

17. S. R. Das and V. D. P. Sastri, “Spectral distribution and color of tropical daylight,”J. Opt. Soc. Am. **55**, 319 (1965); [CrossRef]

17. V. D. P. Sastri and S. R. Das, “Spectral distribution and color of north sky at Delhi,”J. Opt. Soc. Am. **56**, 829 (1966); [CrossRef]

17. “Typical spectral distributions and color for tropical daylight,”J. Opt. Soc. Am. **58**, 391 (1968).

## REFORMULATION OF THE PROBLEM

*m*values of

*∊*

*in Eq. (3) form a (column) vector*

_{i}*∊*that specifies the light

*E*(λ). The

*n*values of

*σ*

*in Eq. (2) form a (column) vector*

_{j}*σ*that specifies the surface reflectance

*S*

*(λ). Substituting Eqs. (2) and (3) into Eq. (1) permits us to express the relationship between the daylight surface reflectances and sensor responses by the matrix equation*

^{x}*ρ*

*is a (column) vector formed from the quantum catches of the*

^{x}*p*sensors at location

*x*. The matrix Λ

*is*

_{∊}*p*by

*n*, and its

*kj*th entry is of the form ∫

*E*(λ)

*S*

*(λ)*

_{j}*R*

*(λ)dλ. The matrix Λ*

_{k}*captures the role of the light in transforming surface reflectances at each location*

_{∊}*x*into sensor quantum catches.

*is known, we see that to recover the*

_{∊}*n*weights that determine the surface reflectance we need merely solve a set of simultaneous linear equations. The recovery procedure reduces to matrix inversion when

*p*=

*n*. If

*p*is less that

*n*Eq. (4) is underdetermined and there is no unique solution.

*∊*or the spectral reflectances even when

*p*=

*n*. The matrix Λ

*is square. For any*

_{∊}*∊*such that Λ

*is nonsingular there is a set of surface reflectances that satisfy Eq. (4). Any such choice of a light vector*

_{∊}*∊*and corresponding surface reflectances

*σ*

*could have produced the observed surface reflectances. No unique solution is possible without additional information concerning lights and surfaces in the scene.*

^{x}*p*>

*n*. Suppose that there are

*p*=

*n*+ 1 linearly independent sensors to sample the image at each location spectrally. In this case from

*s*different spatial locations

*s*(

*n*+ 1) data values are obtained on the left-hand side of the equation. The number of unknown parameters is only

*sn*unknowns from the different surface vectors and

*m*unknowns from the light vector. After sampling at

*s*>

*m*locations we finally obtain more data values than unknowns.

*n*+ 1 sensors can contain enough information to permit exact recovery of both the lighting parameter

*∊*and the surface reflectance

*σ*

*at each location. Next we outline a method for computing the light*

^{x}*∊*and the

*n*-dimensional surface reflectance vector

*σ*

*given the*

^{x}*n*+ 1 dimensional sensor response vector

*ρ*

*at each location.*

^{x}## COMPUTATIONAL METHOD

*∊*and the surface vectors

*σ*

*contribute to the value of the sensor vectors in different ways. The ambient light vector specifies the value of the light transformation matrix, Λ*

^{x}*. The matrix Λ*

_{∊}*is a linear transformation from the*

_{∊}*n*-dimensional space of surface reflectances

*σ*

*into the*

^{x}*n*+ 1-dimensional space of sensor quantum catches

*ρ*

*. The sensor response to any particular surface,*

^{x}*σ*

*, is the weighted sum of the*

^{x}*n*column vectors of Λ

*. Consequently, the sensor responses must fall in a proper subspace of the sensor space determined by Λ*

_{∊}*, and therefore by the lighting parameter*

_{∊}*∊*.

*p*= 3) and two degrees of freedom in the surface reflectances (

*n*= 2). In the particular example shown in Fig. 1, the two-dimensional surface vectors span a plane (passing through the origin) in the three-dimensional sensor space. The light

*∊*determines the plane.

*∊*. Second, once we know the light vector

*∊*, we determine the lighting matrix Λ

*and achieve our goal of recovering the surface vectors simply by inverting this transformation. The exact mathematical conditions that must obtain in order for our procedure to yield the unique correct result are analyzed by Maloney. [18]*

_{∊}*σ*

*in the least-squares sense. [19]*

^{x}19. See Ref. [18], Chap. 4, for details.

## IMPLICATIONS

*E*(λ) and

*E*′(λ). If the lights for which human color constancy succeeds form a linear model, then color appearance should also be preserved when the surfaces are viewed under weighted mixtures of the ambient lights

*E*(λ) and

*E*′(λ).

*p*− 1 distinct surfaces are needed in order to determine uniquely the light vector

*∊*. In the presence of small deviations from the linear models of light and surface reflectances, an increase in the number of distinct surface reflectances will, in general, improve the estimate of the light and the corresponding surface reflectance estimates. Our analysis suggests that color constancy should improve with the number of distinct surfaces in a scene. [20]

21. E. N. Willmer and W. D. Wright, “Colour sensitivity of the fovea centralis,” Nature **156**, 119 (1945); [CrossRef]

21. D. R. Williams, D. I. A. MacLeod, and M. Hayhoe, “Punctate sensitivity of the blue-sensitive mechanism,” Vision Res. **21**, 1357 (1981); [CrossRef] [PubMed]

21. F. M. de Monasterio, S. J. Schein, and E. P. McCrane, “Staining of blue-sensitive cones of the macaque retina by a fluorescent dye,” Science **213**, 1278 (1981) [CrossRef] .

*∊*has been estimated, we may calculate the inverse of the lighting matrix Λ

*. Once this matrix is known, the*

_{∊}*n*+ 1 quantum catches at each location are redundant: only

*n*sensor values are needed to compute the value of

*σ*

*. It follows that if the ambient light varies slowly across the scene, the*

^{x}*n*+ 1th sensor class need not be present at as high a sampling density as the other sensor classes.

## ACKNOWLEDGMENTS

## REFERENCES AND NOTES

1. | See P. Sällström, “Colour and physics: some remarks concerning the physical aspects of human colour vision,” Institute of Physics Rep. 73-09 (University of Stockholm, Stockholm, 1973). |

2. | The computational method that we develop requires only that the ambient light be approximately constant over small local patches of the image. The method is easier to explain if we restrict attention to a region of the image across which the ambient light does not change. |

3. | In general the surface reflectance function may depend on the geometry of the scene, the angle of incidence of the light on the surface, and the angle between the surface and the line of sight. We are concerned here with the analysis of a single image drawn from a scene with fixed geometric relations among objects, light sources, and the visual sensor array. |

4. | D. B. Judd, “Hue saturation and lightness of surface colors with chromatic illumination,” J. Opt. Soc. Am. H. Helson, “Fundamental problems in color vision. I. The principle governing changes in hue saturation and lightness of non-selective samples in chromatic illumination,”J. Exp. Psychol. |

5. | E. H. Land and J. J. McCann, “Lightness and retinex theory,”J. Opt. Soc. Am. E. H. Land, “Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image,” Proc. Nat. Acad. Sci. U.S. E. H. Land, D. H. Hubel, M. Livingston, S. Perry, and M. Burns, “Colour-generating interactions across the corpus callosum,” Nature |

6. | D. Brainard and B. Wandell, “An analysis of the retinex theory of color vision,” Stanford Applied Psychology Lab. Tech. Rep. 1985-04 (Stanford University, Stanford, Calif., 1985). |

7. | G. Buchsbaum, “A spatial processor model for object colour perception,”J. Franklin Inst. |

8. | It is not possible to recover |

9. | M. H. Brill, “A device performing illuminant-invariant assessment of chromatic relations,”J. Theor. Biol. |

10. | W. S. Stiles, G. Wyszecki, and N. Ohta, “Counting metameric object-color stimuli using frequency-limited spectral reflectance functions,”J. Opt. Soc. Am. |

11. | For a discussion of band-limited functions, see R. Bracewell, |

12. | G. Buchsbaum and A. Gottschalk, “Chromaticity coordinates of frequency-limited functions,” J. Opt. Soc. Am. A |

13. | J. Cohen, “Dependency of the spectral reflectance curves of the Munsell color chips,” Psychonomic Sci. |

14. | See K. V. Mardia, J. T. Kent, and J. M. Bibby, |

15. | E. Krinov, details of the fit of Cohen’s characteristic vectors to the Munsell surface reflectances are given in L. Maloney, “Computational approaches to color constancy,” Stanford Applied Psychology Lab. Tech. Rep. 1985-01 (Stanford University, Stanford, Calif., 1985). |

16. | D. B. Judd, D. L. MacAdam, and G. Wyszecki, “Spectral distribution of typical daylight as a function of correlated color temperature,”J. Opt. Soc. Am. |

17. | E. R. Dixon, “Spectral distribution of Australian daylight,”J. Opt. Soc. Am. G. T. Winch, M. C. Boshoff, C. J. Kok, and A. G. du Toit, “Spectroradiometric and calorimetric characteristics of daylight in the southern hemisphere: Pretoria, South Africa,”J. Opt. Soc. Am. S. R. Das and V. D. P. Sastri, “Spectral distribution and color of tropical daylight,”J. Opt. Soc. Am. V. D. P. Sastri and S. R. Das, “Spectral distribution and color of north sky at Delhi,”J. Opt. Soc. Am. “Typical spectral distributions and color for tropical daylight,”J. Opt. Soc. Am. |

18. | L. Maloney, “Computational approaches to color constancy,” Stanford Applied Psychology Lab. Tech. Rep. 1985-01 (Stanford University, Stanford, Calif., 1985). |

19. | See Ref. [18], Chap. 4, for details. |

20. | R. B. MacLeod, “An experimental investigation of brightness constancy,” Arch. Psychol. |

21. | E. N. Willmer and W. D. Wright, “Colour sensitivity of the fovea centralis,” Nature D. R. Williams, D. I. A. MacLeod, and M. Hayhoe, “Punctate sensitivity of the blue-sensitive mechanism,” Vision Res. F. M. de Monasterio, S. J. Schein, and E. P. McCrane, “Staining of blue-sensitive cones of the macaque retina by a fluorescent dye,” Science |

**History**

Original Manuscript: July 18, 1985

Manuscript Accepted: August 9, 1985

Published: January 1, 1986

**Citation**

Laurence T. Maloney and Brian A. Wandell, "Color constancy: a method for recovering surface spectral reflectance," J. Opt. Soc. Am. A **3**, 29-33 (1986)

http://www.opticsinfobase.org/josaa/abstract.cfm?URI=josaa-3-1-29

Sort: Year | Journal | Reset

### References

- See P. Sällström, “Colour and physics: some remarks concerning the physical aspects of human colour vision,” (University of Stockholm, Stockholm, 1973).
- The computational method that we develop requires only that the ambient light be approximately constant over small local patches of the image. The method is easier to explain if we restrict attention to a region of the image across which the ambient light does not change.
- In general the surface reflectance function may depend on the geometry of the scene, the angle of incidence of the light on the surface, and the angle between the surface and the line of sight. We are concerned here with the analysis of a single image drawn from a scene with fixed geometric relations among objects, light sources, and the visual sensor array. Sx(λ) refers to the proportion of light returned from the object toward the sensor array within that fixed geometrical framework.
- D. B. Judd, “Hue saturation and lightness of surface colors with chromatic illumination,” J. Opt. Soc. Am. 30, 2 (1940); H. Helson, “Fundamental problems in color vision. I. The principle governing changes in hue saturation and lightness of non-selective samples in chromatic illumination,”J. Exp. Psychol. 23, 439 (1938). [CrossRef]
- E. H. Land, J. J. McCann, “Lightness and retinex theory,”J. Opt. Soc. Am. 61, 1 (1971); E. H. Land, “Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image,” Proc. Nat. Acad. Sci. U.S. 80, 5163 (1983); E. H. Land, D. H. Hubel, M. Livingston, S. Perry, M. Burns, “Colour-generating interactions across the corpus callosum,” Nature 303, 616 (1983). [CrossRef] [PubMed]
- D. Brainard, B. Wandell, “An analysis of the retinex theory of color vision,” (Stanford University, Stanford, Calif., 1985).
- G. Buchsbaum, “A spatial processor model for object colour perception,”J. Franklin Inst. 310, 1 (1980). [CrossRef]
- It is not possible to recover E(λ) better than to within a multiplicative constant given only the sensor quantum catches. If, for example, the intensity of the light is doubled to 2E but all reflectances are halved to ½Sx(λ), it is easy to verify that the sensor quantum catches in Eq. (1) are unchanged. When we speak of recovering the ambient light and surface reflectances, we mean recovery up to this unknown mutiplicative constant.
- M. H. Brill, “A device performing illuminant-invariant assessment of chromatic relations,”J. Theor. Biol. 71, 473 (1978). [CrossRef] [PubMed]
- W. S. Stiles, G. Wyszecki, N. Ohta, “Counting metameric object-color stimuli using frequency-limited spectral reflectance functions,”J. Opt. Soc. Am. 67, 779 (1977). [CrossRef]
- For a discussion of band-limited functions, see R. Bracewell, The Fourier Transform and Its Application, 2nd ed. (McGraw-Hill, New York, 1978), Chap. 10.
- G. Buchsbaum, A. Gottschalk, “Chromaticity coordinates of frequency-limited functions,” J. Opt. Soc. Am. A 1, 885 (1984). [CrossRef] [PubMed]
- J. Cohen, “Dependency of the spectral reflectance curves of the Munsell color chips,” Psychonomic Sci. 1, 369 (1964).
- See K. V. Mardia, J. T. Kent, J. M. Bibby, Multivariate Analysis (Academic, London, 1979), Chap. 8, for a discussion of characteristic vector analysis (also known as principal-components analysis or the Karhunen–Loève decomposition).
- E. Krinov, Spectral Reflectance Properties of Natural Formations, Technical translation TT-439 (National Research Council of Canada, Ottawa, 1947); details of the fit of Cohen’s characteristic vectors to the Munsell surface reflectances are given in L. Maloney, “Computational approaches to color constancy,” (Stanford University, Stanford, Calif., 1985).
- D. B. Judd, D. L. MacAdam, G. Wyszecki, “Spectral distribution of typical daylight as a function of correlated color temperature,”J. Opt. Soc. Am. 54, 1031 (1964). [CrossRef]
- E. R. Dixon, “Spectral distribution of Australian daylight,”J. Opt. Soc. Am. 68, 437 (1978); G. T. Winch, M. C. Boshoff, C. J. Kok, A. G. du Toit, “Spectroradiometric and calorimetric characteristics of daylight in the southern hemisphere: Pretoria, South Africa,”J. Opt. Soc. Am. 56, 456 (1966); S. R. Das, V. D. P. Sastri, “Spectral distribution and color of tropical daylight,”J. Opt. Soc. Am. 55, 319 (1965); V. D. P. Sastri, S. R. Das, “Spectral distribution and color of north sky at Delhi,”J. Opt. Soc. Am. 56, 829 (1966); “Typical spectral distributions and color for tropical daylight,”J. Opt. Soc. Am. 58, 391 (1968). [CrossRef]
- L. Maloney, “Computational approaches to color constancy,” (Stanford University, Stanford, Calif., 1985).
- See Ref. 18, Chap. 4, for details.
- R. B. MacLeod, “An experimental investigation of brightness constancy,” Arch. Psychol. 135, 1 (1932), reports that human brightness constancy does.
- E. N. Willmer, W. D. Wright, “Colour sensitivity of the fovea centralis,” Nature 156, 119 (1945); D. R. Williams, D. I. A. MacLeod, M. Hayhoe, “Punctate sensitivity of the blue-sensitive mechanism,” Vision Res. 21, 1357 (1981); F. M. de Monasterio, S. J. Schein, E. P. McCrane, “Staining of blue-sensitive cones of the macaque retina by a fluorescent dye,” Science 213, 1278 (1981). [CrossRef] [PubMed]

## Cited By |
Alert me when this paper is cited |

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.

### Figures

Fig. 1 |

« Previous Article | Next Article »

OSA is a member of CrossRef.