OSA's Digital Library

Virtual Journal for Biomedical Optics

Virtual Journal for Biomedical Optics

| EXPLORING THE INTERFACE OF LIGHT AND BIOMEDICINE

  • Editors: Andrew Dunn and Anthony Durkin
  • Vol. 9, Iss. 4 — Apr. 1, 2014
« Show journal navigation

Open problems in color constancy: discussion

C. van Trigt  »View Author Affiliations


JOSA A, Vol. 31, Issue 2, pp. 338-347 (2014)
http://dx.doi.org/10.1364/JOSAA.31.000338


View Full Text Article

Acrobat PDF (224 KB)





Browse Journals / Lookup Meetings

Browse by Journal and Year


   


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools

Share
Citations

Abstract

This paper discusses a number of open problems in color constancy theory whose correct solution is a prerequisite for the theory of the phenomenon. Solutions employing suitable visually meaningful versus physically meaningful basis functions (principal components) are examined. In the former case the starting point is an estimate of the first derivative of the reflectance (illuminant), essential for defining color, instead of an estimate of the reflectance (illuminant), as in the latter. Conceptual consequences are discussed. Mathematical and physical constraints are identified. We compare the results of theories that do or do not ignore them. The following questions are considered. (1) Do unique solutions of the estimation problem exist everywhere in the object-color solid belonging to the illuminant? (2) Are they physically meaningful, i.e., at least nonnegative? (3) Are they representative for reflectance and spectral distribution functions? (4) What role plays metamerism?

© 2014 Optical Society of America

1. INTRODUCTION

The visual system is a signal detection, discrimination, and processing system. Its perceptual variables exhibit color (i.e., appearance to the generic human observer) constancy, the phenomenon that the appearance of a scene is qualitatively preserved across illuminants. Somehow, the visual system manages to construct almost illuminant-independent variables from illuminant-dependent input. A simple consideration sheds some light on the phenomenon.

Let ρ(λ) be the reflectance of some patch, S(λ) the illuminant, E(λ) the equi-energy spectrum equal to unity for all λ, chosen as the reference illuminant, and A(λ) a linear superposition of the CIE color-matching functions, normalized such that the integral of A(λ)E(λ) over the visual range equals unity, for example (bar tacitly understood), x(λ)/XE, y(λ)/YE or z(λ)/ZE, with XE, YE, and ZE the tristimulus values of E(λ). The integral over the visual range of ρ(λ)S(λ)A(λ) over the same integral of S(λ)A(λ), called a von Kries type quotient, is obviously strictly independent of the amplitude of S(λ). Numerical analysis [1

1. C. E. Fröberg, Introduction to Numerical Analysis (Addison-Wesley, 1966), p. 297.

] is aware that such quotients, almost illuminant-independent if ρ(λ) is almost constant, are often weakly dependent on other illuminant parameters, for example, the temperature T when S(λ) is a blackbody radiator. Numerical simulation confirms this. Von Kries type quotients are employed in the definition of CIE La*b* space [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

], apparently rather successfully. If A(λ) is a cone sensitivity we have a classic von Kries quotient arising in the von Kries hypothesis and the associated coefficient rule. Attempts to experimentally verify the hypothesis have failed [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

]; see Section 3.

Despite this fact, Forsyth [3

3. D. A. Forsyth, “A novel algorithm for color constancy,” Int. J. Comput. Vis. 5, 5–35 (1990). [CrossRef]

] concluded from numerical work in machine vision that the coefficient rule has much to recommend it if sensitivities are narrowband. “Narrowband” is a relative concept. A theorem [4

4. C. van Trigt, “Illuminant-dependence of von Kries type quotients,” Int. J. Comput. Vis. 61, 5–30 (2005). [CrossRef]

,5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

] states that a von Kries type quotient equals the integral of ρ(λ)E(λ)A(λ) apart from an illuminant-dependent error, which is small if, in a precisely defined sense, A(λ)E(λ) is narrow compared to ρ(λ) and S(λ)/E(λ), both dimensionless, a fact trivially true if one of the latter is constant as a function of λ (width infinity) or if A(λ) is a δ-function (width zero). Despite its counterintuitive nature, the δ-approximation is of conceptual importance: if (and only if [4

4. C. van Trigt, “Illuminant-dependence of von Kries type quotients,” Int. J. Comput. Vis. 61, 5–30 (2005). [CrossRef]

]) it applies is perfect color constancy possible. Cone sensitivities are narrow functions but narrower, nonnegative functions A(λ) exist [6

6. J. A. C. Yule, Principles of Color Reproduction (Wiley, 1967).

]. The analysis reveals not only the physical basis of color constancy but also when it can fail. Reflectance and spectral distribution functions associated with saturated colors are not broad in the definition of the theorem explaining color constancy failures as observed [7

7. H. E. Ives, “The relation between the color of the illuminant and the color of the illuminated object,” Trans. Illum. Eng. Soc. 7, 62–72 (1912).

]. Empiricism and theory show that von Kries type quotients constitute a reasonable starting point for color constancy theory, attempting to construct visual variables with smaller illuminant- dependent error. In this paper, ρ(λ) and S(λ) are desaturated. Occasionally, results for saturated ρ(λ) are stated.

Wyszecki and Stiles [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

], p. 173, remark that “the science of color has not advanced far enough to deal with this problem (various visual phenomena that contribute significantly to color perception) quantitatively.” Thus, it is a good question [11

11. D. A. Foster, “Color constancy,” Vis. Res. 51, 674–700 (2011). [CrossRef]

] whether the situation today is better and, if not, what is the prevalent challenge. The appearance of a patch with reflectance ρ(λ) under S(λ) is a function of the tristimulus values of ρ(λ)S(λ) and S(λ), and, possibly, other variables (surround). The von Kries approach employs three von Kries type quotients. Variables in color constancy theory need the three von Kries type quotients and the chromaticity coordinates of S(λ). Thus, color science must find three nonlinear functions of at least five variables with optimal discrimination properties that are, hopefully, able to describe lightness and color. This urges modesty. As a first step, we could replace in empirical formulas [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

] von Kries type quotients by theoretical color constant variables and see what happens—provided that color constancy theory is sound. However, some approaches do not differentiate between meaningless and meaningful quantities; see Section 2.A. Matrices are assumed not proved to be nonsingular; see Section 2.B. Assumptions seem untenable from the physical and mathematical point of view; see Section 2.C. Information lacking in some models is gathered by means of methods that are apparently not applied by the visual system; see Section 2.D. With the benefit of hindsight, these problems can possibly be solved within the adopted framework. They constitute as many open problems to adherents.

However, of predominant importance, since then color constancy theory takes a different course, are two conceptual problems. First, it must be decided whether basis functions in estimates of ρ(λ) and S(λ) should be visually or physically meaningful, e.g., principal components and phases of daylight. Visually meaningful basis functions are appropriately defined by means of the color-matching functions. The data processing uses symmetric, positive definite 2×2 matrices whose inversion is trivial and, hence, not beyond the power of a biological system. They ensure that the sets of linear equations defining estimates always have a unique solution; see Section 2.B. In this framework we discern desaturated [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

] and different types of saturated colors [13

13. C. van Trigt, “Smoothest reflectance functions. II. Complete results,” J. Opt. Soc. Am. A 7, 2208–2222 (1990). [CrossRef]

]. Nonnegative estimates, 1 if ρ(λ) is concerned, are obtained by a linear analysis in the former case and by nonlinear methods in the latter, in agreement with mathematical predictions; see Section 2.A. The ensuing visual representation of the “world” differs from the physical one, which is an old philosophical issue; see Section 2.D. Actually, dρ/dλ is estimated. Next, its integration and a suitable integration constant yields the estimate of ρ(λ) with basis functions as the integrals of the functions in the representation of dρ/dλ and the function equal to unity for all λ. This order affords a decisive advantage. Since color conveys information about dρ/dλ, constancy of color benefits from its direct estimation, preferably by a least-squares fit; see Section 2.B. Integration suppresses the illuminant-dependent estimation error. The estimate and the function to be estimated, if smooth, are close. On the other hand, if first ρ(λ) is estimated, the required estimate of dρ/dλ must be obtained by differentiation, amplifying estimation error; see Section 2.C for examples.

Sections 2.A2.D can be read independently. In Section 2.B Eqs. (3a) and (3b), the interim summary and discussion of prior results are necessary and, hopefully, sufficient for understanding the role played by the two approaches to the color constancy problem. The passage proves von Helmholtz’s conjecture [10

10. D. Jameson and L. M. Hurvich, “Essay concerning colour constancy,” Ann. Rev. Psychol. 40, 1–22 (1989). [CrossRef]

] and can be read independently. See Section 3 for experimentally verifiable theoretical predictions. In view of the extensive bibliography in Foster [11

11. D. A. Foster, “Color constancy,” Vis. Res. 51, 674–700 (2011). [CrossRef]

], only supplementary references are provided.

2. OPEN PROBLEMS

A. Nonnegativity

B. Physically versus Visually Meaningful Basis Functions

Like orthogonal polynomials, principal components form a complete set of physically meaningful functions. Any function that is like ρ(λ) can be written as a generally infinite series with appropriate coefficients. A finite expansion with tristimulus values under an estimate of S(λ) equal to those of ρ(λ)S(λ), is called an estimate of ρ(λ), if its values are between 0 and 1. The definition of a similar nonnegative estimate of S(λ), determined by the tristimulus values of S(λ), needs, first, a decision about the appropriate basis functions. The three cone sensitivities are linearly independent on (λb, λe), with λb and λe the begin and end points of the visual range. If desired, we can construct mutually orthogonal functions (apply the Gram–Schmidt method). Functions orthogonal to the color-matching functions multiplied by E(λ) are called metameric blacks [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

] under E. Together, the functions constitute a complete set of visually meaningful basis functions. Any estimate of S(λ) [estimate of ρ(λ)] can be written as a linear superposition of the cone sensitivities multiplied by E(λ) [estimate of S(λ)] plus some metameric black under E(λ) [idem]. An unjustifiable restriction of S(λ) to phases of daylight has been removed. For example, blackbody radiators also possess excellent color rendering properties [16

16. CIE, Method of measuring and specifying colour rendering properties of light sources (Bureau Central de la CIE, 1974).

], which a valid theory should be able to explain. It can be proved that representations are invariant for a nonsingular transformation of the color-matching functions. For practical purposes, we may employ the CIE functions and tristimulus values.

Consider N=3 estimates Se(λ) of S(λ) and ρe(λ) of ρ(λ) to be calculated, respectively, from its cone signals or tristimulus values X0, Y0, Z0, and from those of ρe(λ)Se(λ), corresponding to X, Y, Z of ρ(λ)S(λ). The three physically meaningful basis functions are daylight phases [if Se(λ)] or principal components [if ρe(λ) is concerned]. On the other hand, the visually meaningful ones are the cone sensitivities, multiplied by either E(λ) or Se(λ) in case of Se(λ)/E(λ) or ρe(λ), both dimensionless. In all cases, the coefficients cj, j=1, 2, 3 of the basis functions satisfy an inhomogeneous 3×3 set of linear equations with, on the left-hand side, the given ith cone signal of either Se(λ) or ρe(λ)Se(λ). On the right-hand side, the matrix elements equal the integral over the visual range of the ith visually meaningful basis function and the jth basis function, chosen either visually or physically meaningful. Recall from linear algebra that the solution of the set exists and is unique if the determinant of the matrix is nonzero. If it vanishes, either no solution exists or, if it exists, it is not unique. If this situation is to be avoided, the determinant must not vanish. This is trivial for the visually meaningful representation of Se(λ) and ρe(λ). The matrix involved is symmetric and positive definite [8

8. G. Polya and G. Szegö, Aufgaben und Lehrsätze aus der Analysis, Band II (Springer-Verlag, 1964), aufgabe 63, p. 114.

]. Its determinant, equal to the product of its positive eigenvalues, is nonzero. If this holds true when the phases of daylight or principal components are concerned, its proof seems nontrivial. In more sophisticated approaches [18

18. G. Iverson and M. D’Zmura, “Criteria for color constancy in trichromatic bilinear models,” J. Opt. Soc. Am. A 11, 1970–1975 (1994). [CrossRef]

] the same problem occurs. For the sake of the argument, we grant that the determinant is nonzero. The homogeneous 3×3 system of linear equations has only the solution zero so that a metameric black involving three principal components is zero for all λ. For the evaluation of the matrix elements involving physically meaningful basis functions, the human visual system must dispose of “knowledge of the physical world” that then must reside in memory. This leads to problems connected with the development of color constant vision in infants, who start with an empty memory [19

19. J. I. Dannemiller, “Computational approaches to color constancy, adaptive and ontogenetic considerations,” Psychol. Rev. 96, 255–266 (1989). [CrossRef]

]. The same argument motivates the choice of E(λ) as the reference illuminant instead of D65, for example.

These visually meaningful estimates [20

20. J. B. Cohen and W. E. Kappauf, “Metameric color stimuli, fundamental metamers and Wyszecki’s metameric blacks,” Am. J. Psychol. 95, 537–564 (1982). [CrossRef]

] break down when the cone sensitivities tend to δ-functions because the mentioned matrix elements for i=j, essentially the integral of the product of two δ-functions with the same peak, diverge. Nonnegativity is still to be proved. The estimates, consisting of the three peaked cone sensitivities, are not representative of actual functions, e.g., the constant reflectance. For smoothest estimates ρ0(λ) of ρ(λ), its second derivative is a weighted sum of the visual basis functions, replacing Se(λ) by the smoothest estimate S0(λ) of S(λ), with three coefficients to be determined, see Eq. (4) of Ref. [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

] or Eq. (A4) in Appendix A with w(λ)=1, replacing S(λ) by S0(λ) and, for practical purposes, the cone sensitivities by the CIE color-matching functions. The solution of the inhomogeneous differential equation involved whose first derivative vanishes at both ends of the visual range (two boundary conditions), defines new visually meaningful basis functions and, next, leads to a 3×3 inhomogeneous set of linear equations for its coefficients with symmetric, positive definite matrix A in Eq. (9) of Ref. [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

], hence nonsingular. Its solution ρ0(λ), =ρ(λ), if ρ(λ) is constant, always exists and is independent of the amplitude of S0(λ). It is between 0 and 1 for tristimulus values X, Y, Z in the principal domain of the object color solid belonging to S0(λ)0 for all λ, similarly constructed, see Eq. (16) of Ref. [5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

]. See Fig. 2 of Ref. [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

] for examples of ρ0(λ).

Actually, dρ/dλ and dS/dλ are estimated and, next, ρ(λ) and S(λ) by integration and a suitable integration constant. We prove that dS0/dλ is a least-squares fit to dS/dλ. In Eqs. (A2)–(A9) of Appendix A with w(λ)=1 replace ρ(λ), X, Y, Z, S(λ), X0, Y0, Z0, and μj by, respectively, S0(λ), X0, Y0, Z0, E(λ), XE, YE, ZE, and sj. From Eq. (A9) we have
dS0/dλ=s1[ϕ1(λ)ϕ3(λ)]+s2[ϕ2(λ)ϕ3(λ)].
(1)
[ϕ1(λ),ϕ2(λ),ϕ3(λ)]=λbλ[x(λ)/XE,y(λ)/YE,z(λ)/ZE]E(λ)dλ
replacing the functions fj(λ) in Eq. (A6). Since ϕj(λ)=0 at λb and =1 at λe dS0/dλ=0 there. Next, minimize, as a function of s1 and s2,
(dS/dλdS0/dλ)2dλ,
by differentiating with respect to them and putting the result equal to zero. We obtain after some calculations
[X0/XEZ0/ZE,Y0/YEZ0/ZE]*=C[s1,s2]*,
(2)
where * means transpose, converting a row vector into a column vector. The symmetric and positive definite matrix C has elements for i, j=1, 2:
ci,j=[ϕi(λ)ϕ3(λ)][ϕj(λ)ϕ3(λ)]dλ.
Equation (2) coincides with Eq. (A11), mutatis mutandis. S0(λ) is obtained by integrating from λ to λe with appropriate integration constant S0(λe) [see Eq. (16) of Ref. [5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

]], and is linear in its tristimulus values X0, Y0, Z0. The three basis functions are these integrals of the two functions on the right-hand side of Eq. (1) and the function equal to unity for all λ. The least-squares fit guarantees that S0(λ) is representative of S(λ), in general. Since they are metameric under E, they intersect at the, at least three, zeroes of blacks [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

,21

21. C. van Trigt, “Metameric blacks and estimating reflectance,” J. Opt. Soc. Am. A 11, 1003–1024 (1994). [CrossRef]

] 449, 539, and 608 nm. On (449, 539) and (539, 608) the mean square difference of S(λ) and S0(λ) cannot be large if the mean square difference of dS/dλ and dS0/dλ is sufficiently small by Wirtinger’s inequality [22

22. D. S. Mitrinovic, J. E. Pecaric, and A. M. Fink, Inequalities Involving Functions and Their Integrals and Derivatives (Kluwer Academic, 1991). Wirtinger’s inequality occurs in two versions.

]. On (449, 608) S0(λ) and S(λ) are then close. Near the ends of visual range S0(λ) and S(λ) in general differ as the latter’s boundary values are unknown. In the definition of dρ0/dλ [see Eqs. (A7a) and (A6)], when w(λ)=1, this error propagates favorably since the color-matching functions are zero at these ends. Thus, the error introduced by replacing S(λ) by S0(λ) is small, if S(λ) is smooth. If S0(λ)=S(λ), dρ0/dλ is a least-squares fit to dρ/dλ, by the prior proof. Since dρ0/dλ like dS0/dλ disposes of two parameters only, a least-squares fit is indispensable for constancy of color [9

9. C. van Trigt, “Linear models in color constancy theory,” J. Opt. Soc. Am. A 24, 2684–2691 (2007). [CrossRef]

]. In general, S0(λ)S(λ), especially at the ends of the visual range. Since the error propagates favorably, the mean square difference of dρ0/dλ and dρ/dλ is almost minimal. On (449, 608) ρ0(λ) and ρ(λ) cannot much differ if that difference is sufficiently small, as discussed above. Examples in Fig. 2 of Ref. [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

] show that the terms ρ(λb)ρ0(λb) and ρ(λe)ρ0(λe) are, in general, not small since ρ(λb) and ρ(λe), unknown to the visual system, assume arbitrary values between 0 and 1. We suppress the resulting error.

We have the basic equation on (λb, λe):
ρ(λ)=ρ0(λ)+r(λ),
(3a)
where, summarizing, ρ0(λ) is the smoothest N=3 estimate of ρ(λ), assumed desaturated, with two visually meaningful basis functions dependent on the (chromaticity coordinates of) the smoothest estimate S0(λ) of S(λ), both with tristimulus values X0, Y0, Z0; see Eqs. (A7), simplified in Eq. (A9), w(λ) = 1 and Eq. (16) of Ref. [5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

]. The other basis function equals unity for all λ. The illuminant-dependent coefficients are uniquely determined by the tristimulus values X, Y, Z of ρ0(λ)S0(λ) equal to those of ρ(λ)S(λ); 0ρ0(λ)1 is satisfied if X, Y, Z is in the principal domain [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

] of the object-color solid belonging to S0(λ), a condition satisfied for desaturated ρ(λ). The residual estimation error r(λ), zero for constant ρ(λ), is small on (449, 608) if ρ(λ) is smooth, but, in general, large at the ends of the visual range. Perceptual variables are functions of color constant visual variables. The latter are defined by the left-hand side of the following equation with some narrow A(λ)E(λ). We have
ρ0(λ)E(λ)A(λ)dλ=ρ(λ)E(λ)A(λ)dλr(λ)E(λ)A(λ)dλ.
(3b)
Since A(λ)E(λ) tends fast to zero at the ends of the visual range, the in general large values of r(λ) there are suppressed. We evaluate the remaining error. Define
Q(ρ,S)=ρ(λ)S(λ)A(λ)dλρ(λ)E(λ)A(λ)dλS(λ)A(λ)dλ.
Q(ρ,S) divided by the integral of S(λ)A(λ) measures the difference of the von Kries type quotient and the strictly illuminant-independent second integral on the right; see Eq. (4) of Ref. [5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

]. Consider [Q(ρ,S)Q(ρ0,S0)] over the integral of S(λ)A(λ), equal to the integral of S0(λ)A(λ). Since the von Kries type quotients belonging to ρ(λ), S(λ) and ρ0(λ), S0(λ) are equal, the error in Eq. (3b) equals, apart from a minus sign, the first term. See Eq. (21) of Ref. [5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

] with Se(λ) and ρe(λ) replaced by S0(λ) and ρ0(λ). A theorem (see Eq. (12) of Ref. [5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

]), shows that the error depends on, among other things, dS/dλdS0/dλ and dρ/dλdρ0/dλ, small for smooth S(λ) and ρ(λ). Hence, the visual variable defined by the integral on the left of Eq. (3b), 0 for constant ρ0(λ), is almost illuminant-independent, with error smaller than in the case of the von Kries type quotient. If it is virtually zero, the left-hand side of Eq. (3b) recovers from tristimulus values X, Y, Z of ρ(λ)S(λ) the image as it would have appeared if it had been taken under E, the first integral on the right. If so, this proves von Helmholtz’s conjecture [3

3. D. A. Forsyth, “A novel algorithm for color constancy,” Int. J. Comput. Vis. 5, 5–35 (1990). [CrossRef]

,9

9. C. van Trigt, “Linear models in color constancy theory,” J. Opt. Soc. Am. A 24, 2684–2691 (2007). [CrossRef]

,10

10. D. Jameson and L. M. Hurvich, “Essay concerning colour constancy,” Ann. Rev. Psychol. 40, 1–22 (1989). [CrossRef]

], E(λ) defining “white light.” Equations (3) also apply to saturated ρ(λ). Substitute ρ0(λ), derived in that case [13

13. C. van Trigt, “Smoothest reflectance functions. II. Complete results,” J. Opt. Soc. Am. A 7, 2208–2222 (1990). [CrossRef]

].

Strongly different metameric ρ(λ) possess the same estimate ρ0(λ). Their well-known mismatch [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

] under E causes the residual error in Eq. (3b) to vary among them. Although it is appreciably smaller than in the von Kries case, in general, the experiment [see Eqs. (7)–(9)], could reveal that it is not small enough. However, ρ0(λ) corresponds to the simplest choice w(λ)=1 from a class of smoothest reflectances involving a weight function w(λ)0, yielding similar but not the same results [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

]. Another choice [see Eq. (A10)] could lead to a smaller error in practice or better mimic the human visual system.

Alternative approaches [18

18. G. Iverson and M. D’Zmura, “Criteria for color constancy in trichromatic bilinear models,” J. Opt. Soc. Am. A 11, 1970–1975 (1994). [CrossRef]

,11

11. D. A. Foster, “Color constancy,” Vis. Res. 51, 674–700 (2011). [CrossRef]

] claim that ρ(λ) in Eq. (3a) can be modeled by a finite weighted sum of N=38 physical functions, replacing ρ0(λ), with, essentially, r(λ)=0 everywhere. If the N3 parameters can be uniquely determined (see Section 2.D) the claim ensures that the representation is between 0 and 1 for all λ. It inherits the property from the actual ρ(λ), making the work of Section 2.A superfluous. This obviously crucial claim is discussed next.

C. Finite Models versus Estimates

Physically meaningful basis functions, e.g., orthogonal principal components or polynomials are illuminant-independent. A given sum with three expansion coefficients such that its values are between 0 and 1 can be interpreted as a reflectance ρ(λ). The expansion coefficients are uniquely determined by its cone signals. We have constructed an estimate of ρ(λ) that coincides with ρ(λ) itself. We then say that ρ(λ) possesses a finite model. This example is of interest for showing the correctness of an algorithm [18

18. G. Iverson and M. D’Zmura, “Criteria for color constancy in trichromatic bilinear models,” J. Opt. Soc. Am. A 11, 1970–1975 (1994). [CrossRef]

] that exactly recovers its N3 expansion coefficients and those of S(λ) by multiple views (see Section 2.D), not necessarily the correctness of the underlying mathematical idea. Actually, the issue is whether all or ‘“most” a priori given reflectances in the field of view possess a finite N model. Any such ρ(λ) trivially possesses a finite N model, consisting of ρ(λ), supplemented by N1 orthogonal basis functions from the chosen set. If ρ(λ) is not orthogonal to them, applying the Gram–Schmidt method achieves this. However, we are interested in the validity of the thesis: there exists a common basis such that any ρ(λ) possesses an N model, with N=3,8, dependent on the author, apart from a small amount of residual error, at most, i.e., Eq. (3a) applies with r(λ)0 for all λ. Note the interchange of logical quantifiers, always urging prudence. From “all events [ρ(λ)] possess an (event-dependent) cause [finite model],” it does not follow “there exists a common cause for all events.” This is an important physical and astronomical discovery of the past century, the “Big Bang.”

Recall from mathematics [23

23. E. T. Whittaker and G. N. Watson, A Course of Modern Analysis (Cambridge University, 1962).

]: for any ρ(λ) an integer N, dependent on ρ(λ), exists such that the sum of the first N or less terms from its series expansion, convergent on some interval (λ1, λ2), is equal or close to it. In the latter case, this depends on its rate of uniform convergence on (λ1, λ2), permitting term-by-term integration. Thus the thesis assumes that reflectances constitute a special kind of function such that N is independent of ρ(λ). It is also tacitly assumed that the visual interval (λb, λe) is part of all intervals (λ1, λ2), defined by the physical properties of the different materials corresponding to ρ(λ). This is temporarily accepted. The following two examples show that if a reflectance satisfies a finite, for simplicity, N=3 model in some set of basis functions on (λb, λe) it need not possess a similar finite model in another set of basis functions or the other way around. Apparently, the claimed common set of N basis functions, if it exists, is hard to find.

Define x=[2λλeλb]/(λeλb), mapping (λb, λe) on (1, 1), and consider the two following sets of complete, orthogonal basis functions: (1) Legendre polynomials [24

24. A. Erdelyi, W. Magnus, F. Oberhettinger, and F. G. Tricomi, Higher Transcendental Functions II (McGraw-Hill, 1953), p. 178.

] Pn(x), e.g. P0(x)=1, and (2) Fourier series; basis functions 1, sin(nπx) and cos(nπx), n=1,2. In both systems the constant function equals the first basis function and therefore trivially satisfies the finite n=0, 1, N1 model, as does the function x=P1(x) in system (1). On (1, 1) we have the Fourier series [25

25. Reference [23], p. 161, example 1 on interval (π, +π).

], also easily verified directly, for n=1,2:
01+x=1+2n(1)n+1sin(nπx)/(πn).
(4)

The latter representation is not finite in system (2) and even converges slowly. Similarly, the function sin(πx) satisfies the finite N=3 model in system (2). We have the, also not finite, representation in system (1) for n=0,1:
01+sin(πx)=1+2n(1)n(2n+3/2)J2n+3/2(π)P2n+1(x),
(5)
where Jm denotes a spherical Bessel function of order m=2n+3/2, expressible in elementary functions [24

24. A. Erdelyi, W. Magnus, F. Oberhettinger, and F. G. Tricomi, Higher Transcendental Functions II (McGraw-Hill, 1953), p. 178.

], It can be proved that this series converges fast, yet this convergence speed is of little help.

If ρ(λ) is not equal to a finite weighted sum of N chosen basis functions, the expansion coefficients could be optimally chosen, e.g., such that the estimate is a least-squares fit to ρ(λ), perhaps yielding a sufficiently small amount of residual error as the thesis allows. Since the basis functions of sets (1) and (2) are orthogonal, the truncated series of order N=3 affords this least-squares fit [14

14. G. Szegö, Orthogonal Polynomials (American Mathematical Society, 1967).

]. The basis functions are, respectively Pn(x), n=0, 1, 2 and [27

27. M. D’Zmura, “Color constancy: surface color from changing illumination,” J. Opt. Soc. Am. A 9, 490–493 (1992), Table 1. [CrossRef]

] 1, sin(πx), cos(πx). Equation (4) is reduced to
1+x=1+2sin(πx)/π+residual Fourier error
(= the Fourier series summed from n=2 to infinity). Equation (5) yields
1+sin(πx)=1+3x/π+residual polynomial error
(= the Legendre series summed from 2n+1=3 to infinity). In the two cases, divide both sides of the equation by 2. The left-hand sides can be interpreted as reflectances ρ(λ) modeled by the first two terms on the right-hand sides, whose values are between 0 and 1, as well. Verify that, in both cases, the residual error is nonnegligible. Color conveys information about dρ/dλ. Differentiation of both sides in both cases shows that the residual estimation error is (strongly) amplified, as mathematics predicts.

Although the two simple examples render the prospects of the thesis bleak, they do not rule out that a “special” common finite basis exists. If reflectances ρm(λ), m=1M, M very large compared to N, max[ρm(λ)]=1, all possess a common N finite model in some set of basis functions on a common interval (λ1, λ2), defined by the physics of the different materials with reflectance ρm(λ), the correlation function, defined for λ, λ on that interval,
R(λ,λ)=mρm(λ)ρm(λ)/M
is able to establish that. A claim that all (λ1, λ2) equal (0, ) needs proof. Obviously, the thesis is useless if (λb, λe) is not in the intersection of the M intervals, each belonging to some ρm(λ). Grant this: all eigenfunctions (principal components) of the integral equation with kernel R(λ,λ) are in the N-dimensional space spanned by the basis functions; therefore, N+1 eigenfunctions are linearly dependent. Data [28

28. J. P. S. Parkkinen, J. Hallikainen, and T. Jaaskelainen, “Characteristic spectra of Munsell colors,” J. Opt. Soc. Am. A 6, 318–322 (1989), Fig. 1. [CrossRef]

] show that the first four eigenfunctions are not linearly dependent; therefore, N4. If an analysis would show that linear dependence occurs for, e.g., N+1=8, the N original basis functions can be replaced by N principal components, without loss of generality.

Unfortunately, the analysis is little rewarding unless the set {ρm(λ)} is an unbiased sample from all possible reflectances. A rough calculation [29

29. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982), p. 196.

] suggests that metamerism is a quite common phenomenon for desaturated ρ(λ). Reflectances ρm(λ) from color atlases are usually smooth. However, in practice, metameric, nonsmooth functions also occur, as lamp manufacturers are aware. We could modify R(λ,λ) by replacing any single ρm(λ) by a suitable sum of its metamers, rendering the set unbiased, and start again.

As four principal components define a metameric black, an N4 model of ρ(λ) is reducible to a weighted sum of three principle components and (N3) metameric blacks under S(λ). In principle, multiple views under S(λ) and, at least, one additional illuminant S(λ), all represented by a common, finite N, e.g., =3, model, can recover all coefficients; see Section 2.D. Unless S(λ) and S(λ) strongly differ, a black under S(λ) is almost black under S(λ). It is then doubtful whether its coefficient can reliably be recovered. If the sum of blacks is considered as noise, an acceptable thesis, in this framework, could be: on (λb, λe) any ρ(λ) is represented by an estimate involving three principal components apart from a residual error r(λ), i.e., the sum of blacks under S(λ), not small compared to the estimate in general. This thesis leads to an equation similar to Eq. (3a) with ρ0(λ) replaced by the present estimate with one crucial exception: we cannot evade finding conditions on which the estimate of ρ(λ) is between 0 and 1 for all λ (see Section 2.A) as is done by claiming that ρ(λ) possesses a finite N (necessarily4) model, that inherits this property from the actual ρ(λ). Prospects seem bleak; see Section 2.A.

D. Lack of Information

Economical data processing can free information for the determination of the chromaticity coordinates of S(λ). Consider the light ρ(λ)S(λ) from a scene in three-dimensional space and its image on the retina. Regions on which ρ(λ)S(λ) continuously varies within a few just-noticeable differences as a function of space variables are called patches with tristimulus values equal to their mean value. Distinct borders belong to different patches. The traveling salesman algorithm finds the shortest circuit such that the salesman visits M cities (patches) once and returns to the starting city (patch). We use a perceptually meaningful metric, expressing distances in terms of just-noticeable differences. By deleting the largest edge in the circuit we obtain a chain of M minimally distinct patches. A patch and its successor define, in total, (M1) differences of triplets of tristimulus values associated with reflectance differences Δρ(λ) or different illumination conditions (due to surface curvature, stray light, etc.). One triplet of tristimulus values is freed for the determination of the chromaticity coordinates of S(λ), i.e., the white point. Unfortunately, the complexity of the traveling salesman problem increases fast with M. In any event, differences Δρ(λ) are essential for the discrimination problem so that a decision on the proper definition must be deferred until then. The estimate of Δρ(λ) is obtained by replacing in the theory ρ(λ) by [1+Δρ(λ)]/2, achieving that 1 the estimate of Δρ(λ)1. Color constant visual signals follow from Eq. (3b). We start with the hypothesis that the illumination conditions do not vary across the scene, to be revoked if results are inconsistent or “odd.” “Inconsistent” means that not all M free choices of the triplet (patch) yield the same chromaticity coordinates of S(λ).

Unfortunately, if ρ(λ) and S(λ) are the actual reflectance of some patch and the actual illuminant, respectively, the light ρ(λ)S(λ) incident on the eye can be written for any fixed 0<ξ(λ)1 for all λ as the product of a phenomenological reflectance ρ(λ)ξ(λ) and phenomenological illuminant S(λ)/ξ(λ), corresponding to physically different scenes, visually indiscernible from the actual one. In particular, any patch with reflectance ρ(λ)ξ(λ) can serve as a white point corresponding to the phenomenological reflectance=A for all λ, 0<A1 and similar illuminant ρ(λ)S(λ)/A. “Reality,” i.e., the appearance of the world, is then a construct of the observer’s mind (which designates in some way the white patch; hence. the illuminant), a philosophical thesis (Bishop Berkeley’s famous dictum “esse est percipi”) also defended by Kant and influential among German physiologists at the end of the 19th century [30

30. H. Lang, “Color vision theories in the nineteenth century Germany between idealism and empiricism,” Color Res. Appl. 12, 270–281 (1987). [CrossRef]

].

There exist many visually indiscernible physical worlds but only one observer able to define a common framework. In the basic Eqs. (1)–(3) of Ref. [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

], replace ρ(λ) by ρ(λ)ξ(λ) and S(λ) by S(λ)/ξ(λ). Their solution is the smoothest estimate ρ0(ξ;λ) of ρ(λ)ξ(λ), a weighted sum of the same visually meaningful basis functions as in the case ξ(λ)=1 for all λ (see Eq. (4) of Ref. [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

]), provided that the estimate S0(ξ;λ) of S(λ)/ξ(λ) is a weighted sum of the visually meaningful basis functions of Section 2.B [ξ(λ)=1]. If the tristimulus values of S(λ)/ξ(λ) were known, the present coefficients s1 and s2 in Eq. (2) of this paper and the integration constant S0(ξ;λe) could be calculated as before and, next, the present coefficients of ρ0(ξ;λ) by imposing, as in the case ξ(λ)=1:
ρ(λ)S(λ)A(λ)dλ=ρ0(ξ;λ)S0(ξ;λ)A(λ)dλ.
(6a)
Consider some patch among the M patches with estimates ρ0(ξ;λ) and S0(ξ;λ). It is not our intention to construct ρ0(ξ;λ), but to use the tristimulus values X, Y, Z on the left for the calculation of S0(ξ;λ)0, i.e., the coefficients S0(ξ;λe)0, s1 and s2 in its representation, up to a multiplicative constant, actually two variables. Which spectral distribution function S0(ξ;λ) estimates is, temporarily, of no concern. The surface of the object-color solid belonging to some spectral distribution function is determined by its behavior as a function of λ. The reflectances determining the surface are Schrödinger’s optimal filters Os(λ), for simplicity of Type 1, equal to zero or unity with, at most, two transition wavelengths λ1 and λ2; see p. 181 of Ref. [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

]. In practice, optimal colors are seldom present in the field of view. Theoretically, they are ubiquitous. For any ρ0(ξ;λ) a filter Os(λ) exists, up to a multiplicative factor, metameric under S0(ξ;λ), i.e., we have
ρ0(ξ;λ)S0(ξ;λ)A(λ)dλ=Yλ1λ2S0(ξ;λ)A(λ)dλ/Ys.
(6b)
λ1 and λ2 are determined by the chromaticity of S0(ξ;λ) and the patch; see Eq. (6a) and Figs. 3 (3.7) and 4 (3.7) of Ref. [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

]. Y is its luminance equal to the value of the integral on the left for A(λ)=y(λ). Ys is the maximum attainable luminance, i.e., the value of the integral on the right-hand side. Equation (6b) is then an identity. The cases A(λ)=x(λ) and =z(λ) provide two nonlinear equations for the coefficients S0(ξ;λe), s1 and s2, up to a multiplicative constant, to be solved on the condition S0(ξ;λ)0 for all λ. Although they are difficult, optimal filters Os(λ), i.e., λ1 and λ2, are easily constructed numerically. The entire analysis is no more than an application of the principle that S(λ) [ρ(λ)] can be estimated from given tristimulus values if ρ(λ), e.g., an optimal color, only dependent on two parameters and excluding metamerism, [S(λ)] is known. If the M choices of patches do not yield the same answer, we must drop the hypothesis that the illumination is the same across the image and partition it accordingly.

3. EXPERIMENT AND THEORETICAL PREDICTIONS

Color constancy theory in Section 2.B assigns predominant importance to the estimation of dρ/dλ, determining color. Its estimate dρ0/dλ determines its constancy; see Section 2.B. Wyszecki and Stiles [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

] state that opponent color matching, except unique red, is independent of luminance. Both theory and experiment suggest that disentanglement of color and luminance matching is possible. If so, we focus on constancy of color with the advantage that matches of R/G and Y/B variables [31

31. L. M. Hurvich, Color Vision (Sinauer, 1981).

] can be determined with great precision. An experiment that falsifies this part of the theory falsifies it entirely. Theoretical considerations about the limitations of color constancy shed light on the problem.

The starting point is the discussion of asymmetric matching in Wyszecki and Stiles [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

]. In the interpretation of the experiment concerning human color constancy, we compare the following:
  • (1) The appearance of some ρ(λ) under S(λ), defined by the tristimulus values X, Y, Z of ρ(λ)S(λ). The tristimulus values X0, Y0, Z0 of S(λ) are provided by the background.
  • (2) The corresponding experimental asymmetric appearance match under the reference illuminant E, defined by its tristimulus values X, Y, Z. The tristimulus values of E are XE, YE, ZE (Wyszecki and Stiles [2

    2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

    ] mentions caveats).
  • (3) The appearance of ρ(λ) under E, determined by the calculated tristimulus values of ρ(λ)E(λ).
Appearances under S(λ) and E(λ) are equal if points (2) and (3) coincide. In practice, the roles of (1) and (3) are reversed. Mostly, the reference illuminant is not E. In that case, e.g., D65, we equate the matches under E determined by the tristimulus values of ρ(λ) under S(λ) and under D65; see p. 434 of Ref. [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

]. If ρ(λ) (taken from some color atlas) is replaced by reflectances (taken from other color atlases), metameric to it under S(λ), thus with the same appearance under S(λ), points (1) and (2) are preserved. Due to the mismatch of metamers under E(λ), instead of point (3) we obtain a cloud of points with some center, replacing point (3), and radius, i.e., an error bar, in practice ≈ a few just-noticeable differences. If the asymmetric match is inside the cloud, i.e., if the distance of the center to point (2) over the radius is <1, a reflectance exists that preserves its appearance, even if variants of the Brunswik ratio [11

11. D. A. Foster, “Color constancy,” Vis. Res. 51, 674–700 (2011). [CrossRef]

,32

32. E. Brunswick, “Zur Entwicklung der Albedowahrneming,” Z. Psychol. 64, 216–227 (1928).

] based on a single ρ(λ), denies this. This is in agreement with the, in principle, qualitative nature of color constancy.

We have from Eq. 3 (5.12.1) of Ref. [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

], stating that if an asymmetric match exists, a nonsingular 3×3 matrix TS exists such that
TS[X/X0,Y/Y0,Z/Z0]*=[X/XE,Y/YE,Z/ZE]*,
(7)
where * means transpose, converting a row vector into a column vector. The subscript of TS indicates that the matrix elements ti,j, i, j=1, 2, 3 depend on “adaptation conditions,” to be specified by theory and verified by experiment. Since the reference is fixed, they depend only on S(λ). Instead of tristimulus values we can use cone signals, suitably adapting TS as we will do freely. The matching vector on the right-hand side of Eq. (7) is in the object-color solid of E(λ), if X, Y, Z are the tristimulus values of ρ(λ)E(λ), for some reflectance ρ(λ). If so, the vector elements on the right-hand side of Eq. (7) are von Kries type quotients. In an experiment that uses paper matches, this condition is satisfied. If some patch with reflectance ρ(λ) preserves its appearance under S(λ) when viewed under E(λ), then ρ(λ)=ρ(λ) up to a metameric black that vanishes under E(λ) . If we may ignore the error Q(ρ,S) over the integral of S(λ)Ai(λ) for desaturated, smooth ρ(λ) and S(λ) [see the discussion of Eq. (3b)] we have for narrow Ai(λ)E(λ), i=1, 2, 3,
ρ(λ)S(λ)Ai(λ)dλ/S(λ)Ai(λ)dλ=ρ(λ)E(λ)Ai(λ)dλ.
(8)
A(λ) equal to the cone sensitivities, henceforth tacitly understood, yields a special case of the von Kries hypothesis. Ts equals the diagonal matrix with 1 on the diagonal. Experimental support [2

2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

] for the hypothesis lacks, as the theoretical, relative error [5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

], although zero for the constant reflectance, can amount to some 10%. The color constancy result in Eq. (3b) yields
ρ0(λ)E(λ)Ai(λ)dλ=ρ(λ)E(λ)Ai(λ)dλ,
(9)
apart from a much smaller error than in Eq. (8); see the discussion of Eq. (3b). Recall from Eq. (3a) that ρ0(λ) is a weighted sum of the function equal to unity for all λ and two visually meaningful basis functions dependent on the estimate S0(λ); see Eq. (16) of Ref. [5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

]. Numerical integration on the left of Eq. (9) can be carried out. Substitution of the coefficients in the representation of ρ0(λ), i.e., an illuminant-dependent, weighted sum of von Kries type quotients X/X0, Y/Y0, and Z/Z0 or similar cone signals, shows that the left-hand sides of Eqs. (7) and (9) are of the same form. The elements of Ts, in general nondiagonal, depend on the chromaticity coordinates of S0(λ), equal to those of the actual S(λ). They specify “adaptation conditions.”

If so, Eq. (10) is able to correct a systematic error; dρ0/dλ vanishes at both λb and λe, while this seldom happens for dρ/dλ in practice. Furthermore, ρ0(λ) is smoother than any actual ρ(λ) it estimates. A slightly less smooth estimate is more representative for ρ(λ) in practice; dρ0/dλ is orthogonal to dR1/dλ and dR2/dλ, nonzero at λb and λe, respectively. The integral of (dρe/dλ)2 is the sum of two quadratic expressions. It suffices to construct a smooth estimate whose first derivatives are nonzero at both λb and λe, corresponding to nonzero A and B in Eq. (9). Recall from the discussion of Eqs. (1)–(3) of Ref. [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

] that another class of smoothest estimates is defined by a weight function w(λ)>0, multiplying (dρ/dλ)2 there. The simplest choice w(λ)=1 for all λ yields ρ0(λ) in this paper. It is proved in Appendix A that a visually meaningful w(λ) can be constructed that tends sufficiently fast to zero at λb and λe and thereby achieves that the first derivative of the associated estimate is nonzero there; see Eqs. (A5) and (A8). Since the improved estimate is metameric to ρ0(λ), it is of the form of ρe(λ) with one important exception: as before, we are able to find the slightly different conditions on which it is between 0 and 1 for all λ.

4. CONCLUSION

In its youth, any field of science faces the necessity of finding a consistent, mathematically and physically correct basis. This paper discusses the issue for color constancy. Since it is a qualitative phenomenon, theory can achieve, at most, that color constancy errors, e.g., due to metamerism, usually are small and random. Numerical simulations can establish this. Unfortunately, the issue complicates the experiment that, next, must decide whether the theory adequately mimics the behavior of the human visual system. Theoretically optimal, artificial systems may be of independent interest. Faites vos jeux (choose and play)!

APPENDIX A

In this appendix we construct for a given [estimate, e.g., S0(λ) of the] illuminant S(λ) with tristimulus values X0, Y0, Z0, the smoothest reflectance ρ(λ) associated with a weight function w(λ)>0 for all λ, except, possibly, at λb and λe. The special case w(λ)=1 yields ρ0(λ) in this paper. We are interested in a visually meaningful w(λ) that tends sufficiently fast to zero at λb and λe, thereby achieving that dρ/dλ is nonzero there; see Eq. (A5). We also show that all data processing can be carried out by a trivial inversion of symmetric, positive definite 2×2 matrices instead of a such-like 3×3 matrices, as previously. The needed generalization of Eqs. (1)–(3) of Ref. [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

] reads
0ρ(λ)1,
(A1)
ρ(λ)S(λ)[x(λ),y(λ),z(λ)]dλ=[X,Y,Z],
(A2)
(dρ/dλ)2w(λ)dλ=minimal.
(A3)

The new equations lead to minor modifications of the earlier analysis [12

12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

]. Calculus of variations shows that the solution of Eqs. (A2) and (A3) satisfies
D[w(λ)Dρ]+[μ1x(λ)/X0+μ2y(λ)/Y0+μ3z(λ)/Z0]S(λ)=0,
(A4)
where D=d/dλ. The coefficients μj, so-called Lagrange multipliers, are determined later. The two boundary conditions are
w(λ)dρ/dλ=0atλbandλe.
(A5)
Define
[f1(λ),f2(λ),f3(λ)]=λbλ[x(λ)/X0,y(λ)/Y0,z(λ)/Z0]S(λ)dλ.
(A6)
We have fj(λe)=1 for j=1, 2 and 3 and, from Eq. (A4),
w(λ)dρ/dλ=jμjfj(λ),
(A7a)
ρ(λ)=ρ(λe)+jμjλλefj(λ)dλ/w(λ).
(A7b)
ρ(λe) is to be determined later. We have, from Eq. (A5), at λe
μ3=(μ1+μ2),
(A8)
and, from Eq. (A7a),
w(λ)dρ0/dλ=μ1[f1(λ)f3(λ)]+μ2[f2(λ)f3(λ)].
(A9)
Equation (A7a) enforces dρ0/dλ0 at λb if w(λ) tends to zero at λb in a similar way as the functions fj(λ) do. Suppose that, for λλb,
x(λ)+y(λ)+z(λ)A(λλb)α
for some A, α>0. Substitution of this behavior into Eq. (A6) and integration by parts yields for λλb, i=1 and similarly, i=2, 3:
f1(λ)(λλb)x(λ)S(λ)/[(α+1)X0].

The zero of fi(λ) at λb has multiplicity α+1. Consider the 3×3 determinant D(λ), first row x(λ), y(λ), z(λ); second row x (λb), y (λb), z (λb); third row x (λe), y (λe), z (λe), and the determinant D equal to D(λ), except that in the first row we have XE, YE, ZE. Define (see Eq. (B1) of Ref. [21

21. C. van Trigt, “Metameric blacks and estimating reflectance,” J. Opt. Soc. Am. A 11, 1003–1024 (1994). [CrossRef]

])
w(λ)=E(λ)D(λ)/D>0;λλbandλe.

The integral of w(λ) is unity. The right-hand side of Eq. (A9) equals [x(λ)+y(λ)+z(λ)]/[XE+YE+ZE] multiplied by a function, only dependent on chromaticity coordinates, that possesses a simple zero at λb and, hence, tends to zero in a way similar to fi(λ) for λλb. Hence, we achieve dρ/dλ0 at λb and, similarly, at λe. Evaluation of the determinants shows that w(λ)/E(λ) equals the narrowest, nonnegative function [5

5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

] constructed by Yule [6

6. J. A. C. Yule, Principles of Color Reproduction (Wiley, 1967).

]:
M2(λ)=0.5088x(λ)/XE+1.4088y(λ)/YE+0.1000z(λ)/ZE.
(A10)
We prove that the coefficients μ in Eqs. (A7) are determined by a 2×2 set of linear equations. Multiply Eq. (A7b) by the functions [x(λ)/X0z(λ)/Z0] S(λ) and [y(λ)/Y0z(λ)/Z0] S(λ), whose integral over the visual range is zero. Integration on the left-hand side and Eq. (A2) yields X/X0Z/Z0 and Y/Y0Z/Z0, respectively. Integration by parts in the second term yields
[X/X0Z/Z0,Y/Y0Z/Z0]*=C[μ1,μ2]*,
(A11)
where * means transpose, converting a row vector into a column vector. C is the symmetric, positive definite 2×2 matrix with elements
ci,j=[fi(λ)f3(λ)][fj(λ)f3(λ)]dλ/w(λ).

Next, we multiply Eq. (A7b) by S(λ)z(λ)/Z0 and integrate over the visual range. By Eq. (A2), the left-hand side is Z/Z0. On the right-hand side, the first term is ρ(λe). The second term is calculated by using integration by parts, applying Eq. (A7b) and eliminating the coefficients μ by Eq. (A11). Define coefficients νj such that
ρ(λe)=ν1X/X0+ν2Y/Y0+ν3Z/Z0;jνj=1.
(A12)
We obtain
[ν1,ν2]*=C1[f1,f2]*;fi=[fi(λ)f3(λ)]f3(λ)dλ/w(λ).

Similarly, we obtain, with gj(λ)=1fj(λ),
ρ(λb)=ν1X/X0+ν2Y/Y0+ν3Z/Z0;jνj=1,
[ν1,ν2]*=C1[g1,g2]*;gi=[gi(λ)g3(λ)]g3(λ)dλ/w(λ).

The signal processing needs no more than (the trivial inversion of) 2*2 matrices. The coefficients ν and v define the principal domain, comprising X, Y, Z in Eq. (A2) such that Eq. (A1) is satisfied for desaturated ρ(λ).

REFERENCES AND NOTES

1.

C. E. Fröberg, Introduction to Numerical Analysis (Addison-Wesley, 1966), p. 297.

2.

G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).

3.

D. A. Forsyth, “A novel algorithm for color constancy,” Int. J. Comput. Vis. 5, 5–35 (1990). [CrossRef]

4.

C. van Trigt, “Illuminant-dependence of von Kries type quotients,” Int. J. Comput. Vis. 61, 5–30 (2005). [CrossRef]

5.

C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]

6.

J. A. C. Yule, Principles of Color Reproduction (Wiley, 1967).

7.

H. E. Ives, “The relation between the color of the illuminant and the color of the illuminated object,” Trans. Illum. Eng. Soc. 7, 62–72 (1912).

8.

G. Polya and G. Szegö, Aufgaben und Lehrsätze aus der Analysis, Band II (Springer-Verlag, 1964), aufgabe 63, p. 114.

9.

C. van Trigt, “Linear models in color constancy theory,” J. Opt. Soc. Am. A 24, 2684–2691 (2007). [CrossRef]

10.

D. Jameson and L. M. Hurvich, “Essay concerning colour constancy,” Ann. Rev. Psychol. 40, 1–22 (1989). [CrossRef]

11.

D. A. Foster, “Color constancy,” Vis. Res. 51, 674–700 (2011). [CrossRef]

12.

C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]

13.

C. van Trigt, “Smoothest reflectance functions. II. Complete results,” J. Opt. Soc. Am. A 7, 2208–2222 (1990). [CrossRef]

14.

G. Szegö, Orthogonal Polynomials (American Mathematical Society, 1967).

15.

J. N. Lythgoe, The Ecology of Vision (Clarendon, 1997).

16.

CIE, Method of measuring and specifying colour rendering properties of light sources (Bureau Central de la CIE, 1974).

17.

J. M. Troost and C. M. M. de Weert, “Techniques for simulating object color under changing illuminant conditions on electronic displays,” Color Res. Appl. 17, 316–327 (1992). [CrossRef]

18.

G. Iverson and M. D’Zmura, “Criteria for color constancy in trichromatic bilinear models,” J. Opt. Soc. Am. A 11, 1970–1975 (1994). [CrossRef]

19.

J. I. Dannemiller, “Computational approaches to color constancy, adaptive and ontogenetic considerations,” Psychol. Rev. 96, 255–266 (1989). [CrossRef]

20.

J. B. Cohen and W. E. Kappauf, “Metameric color stimuli, fundamental metamers and Wyszecki’s metameric blacks,” Am. J. Psychol. 95, 537–564 (1982). [CrossRef]

21.

C. van Trigt, “Metameric blacks and estimating reflectance,” J. Opt. Soc. Am. A 11, 1003–1024 (1994). [CrossRef]

22.

D. S. Mitrinovic, J. E. Pecaric, and A. M. Fink, Inequalities Involving Functions and Their Integrals and Derivatives (Kluwer Academic, 1991). Wirtinger’s inequality occurs in two versions.

23.

E. T. Whittaker and G. N. Watson, A Course of Modern Analysis (Cambridge University, 1962).

24.

A. Erdelyi, W. Magnus, F. Oberhettinger, and F. G. Tricomi, Higher Transcendental Functions II (McGraw-Hill, 1953), p. 178.

25.

Reference [23], p. 161, example 1 on interval (π, +π).

26.

Reference [24], p. 213, formula (7) with λ=1/2, take the imaginary part on both sides, apply p. 179, formula (3) and p. 78, formula (7).

27.

M. D’Zmura, “Color constancy: surface color from changing illumination,” J. Opt. Soc. Am. A 9, 490–493 (1992), Table 1. [CrossRef]

28.

J. P. S. Parkkinen, J. Hallikainen, and T. Jaaskelainen, “Characteristic spectra of Munsell colors,” J. Opt. Soc. Am. A 6, 318–322 (1989), Fig. 1. [CrossRef]

29.

G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982), p. 196.

30.

H. Lang, “Color vision theories in the nineteenth century Germany between idealism and empiricism,” Color Res. Appl. 12, 270–281 (1987). [CrossRef]

31.

L. M. Hurvich, Color Vision (Sinauer, 1981).

32.

E. Brunswick, “Zur Entwicklung der Albedowahrneming,” Z. Psychol. 64, 216–227 (1928).

OCIS Codes
(330.0330) Vision, color, and visual optics : Vision, color, and visual optics
(330.1690) Vision, color, and visual optics : Color
(330.1720) Vision, color, and visual optics : Color vision
(330.1715) Vision, color, and visual optics : Color, rendering and metamerism

ToC Category:
Vision, Color, and Visual Optics

History
Original Manuscript: April 23, 2013
Revised Manuscript: November 3, 2013
Manuscript Accepted: November 22, 2013
Published: January 23, 2014

Virtual Issues
Vol. 9, Iss. 4 Virtual Journal for Biomedical Optics

Citation
C. van Trigt, "Open problems in color constancy: discussion," J. Opt. Soc. Am. A 31, 338-347 (2014)
http://www.opticsinfobase.org/vjbo/abstract.cfm?URI=josaa-31-2-338


Sort:  Author  |  Year  |  Journal  |  Reset  

References

  1. C. E. Fröberg, Introduction to Numerical Analysis (Addison-Wesley, 1966), p. 297.
  2. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982).
  3. D. A. Forsyth, “A novel algorithm for color constancy,” Int. J. Comput. Vis. 5, 5–35 (1990). [CrossRef]
  4. C. van Trigt, “Illuminant-dependence of von Kries type quotients,” Int. J. Comput. Vis. 61, 5–30 (2005). [CrossRef]
  5. C. van Trigt, “Von Kries versus color constancy,” Color Res. Appl.35, 164–183 (2010). A(λ)E(λ) and S(λ)/E(λ) in the present paper are denoted there by A(λ) and S(λ). [CrossRef]
  6. J. A. C. Yule, Principles of Color Reproduction (Wiley, 1967).
  7. H. E. Ives, “The relation between the color of the illuminant and the color of the illuminated object,” Trans. Illum. Eng. Soc. 7, 62–72 (1912).
  8. G. Polya and G. Szegö, Aufgaben und Lehrsätze aus der Analysis, Band II (Springer-Verlag, 1964), aufgabe 63, p. 114.
  9. C. van Trigt, “Linear models in color constancy theory,” J. Opt. Soc. Am. A 24, 2684–2691 (2007). [CrossRef]
  10. D. Jameson and L. M. Hurvich, “Essay concerning colour constancy,” Ann. Rev. Psychol. 40, 1–22 (1989). [CrossRef]
  11. D. A. Foster, “Color constancy,” Vis. Res. 51, 674–700 (2011). [CrossRef]
  12. C. van Trigt, “Smoothest reflectance functions. I. Definition and main results,” J. Opt. Soc. Am. A 7, 1891–1904 (1990). [CrossRef]
  13. C. van Trigt, “Smoothest reflectance functions. II. Complete results,” J. Opt. Soc. Am. A 7, 2208–2222 (1990). [CrossRef]
  14. G. Szegö, Orthogonal Polynomials (American Mathematical Society, 1967).
  15. J. N. Lythgoe, The Ecology of Vision (Clarendon, 1997).
  16. CIE, Method of measuring and specifying colour rendering properties of light sources (Bureau Central de la CIE, 1974).
  17. J. M. Troost and C. M. M. de Weert, “Techniques for simulating object color under changing illuminant conditions on electronic displays,” Color Res. Appl. 17, 316–327 (1992). [CrossRef]
  18. G. Iverson and M. D’Zmura, “Criteria for color constancy in trichromatic bilinear models,” J. Opt. Soc. Am. A 11, 1970–1975 (1994). [CrossRef]
  19. J. I. Dannemiller, “Computational approaches to color constancy, adaptive and ontogenetic considerations,” Psychol. Rev. 96, 255–266 (1989). [CrossRef]
  20. J. B. Cohen and W. E. Kappauf, “Metameric color stimuli, fundamental metamers and Wyszecki’s metameric blacks,” Am. J. Psychol. 95, 537–564 (1982). [CrossRef]
  21. C. van Trigt, “Metameric blacks and estimating reflectance,” J. Opt. Soc. Am. A 11, 1003–1024 (1994). [CrossRef]
  22. D. S. Mitrinovic, J. E. Pecaric, and A. M. Fink, Inequalities Involving Functions and Their Integrals and Derivatives (Kluwer Academic, 1991). Wirtinger’s inequality occurs in two versions.
  23. E. T. Whittaker and G. N. Watson, A Course of Modern Analysis (Cambridge University, 1962).
  24. A. Erdelyi, W. Magnus, F. Oberhettinger, and F. G. Tricomi, Higher Transcendental Functions II (McGraw-Hill, 1953), p. 178.
  25. Reference [23], p. 161, example 1 on interval (−π, +π).
  26. Reference [24], p. 213, formula (7) with λ=1/2, take the imaginary part on both sides, apply p. 179, formula (3) and p. 78, formula (7).
  27. M. D’Zmura, “Color constancy: surface color from changing illumination,” J. Opt. Soc. Am. A 9, 490–493 (1992), Table 1. [CrossRef]
  28. J. P. S. Parkkinen, J. Hallikainen, and T. Jaaskelainen, “Characteristic spectra of Munsell colors,” J. Opt. Soc. Am. A 6, 318–322 (1989), Fig. 1. [CrossRef]
  29. G. Wyszecki and W. S. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae (Wiley, 1982), p. 196.
  30. H. Lang, “Color vision theories in the nineteenth century Germany between idealism and empiricism,” Color Res. Appl. 12, 270–281 (1987). [CrossRef]
  31. L. M. Hurvich, Color Vision (Sinauer, 1981).
  32. E. Brunswick, “Zur Entwicklung der Albedowahrneming,” Z. Psychol. 64, 216–227 (1928).

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited