OSA's Digital Library

Virtual Journal for Biomedical Optics

Virtual Journal for Biomedical Optics


  • Editor: Gregory W. Faris
  • Vol. 5, Iss. 10 — Jul. 19, 2010

Comparison of naïve and expert subjects in the assessment of small color differences

Renzo Shamey, Lina M. Cárdenas, David Hinks, and Roger Woodard  »View Author Affiliations

JOSA A, Vol. 27, Issue 6, pp. 1482-1489 (2010)

View Full Text Article

Enhanced HTML    Acrobat PDF (660 KB)

Browse Journals / Lookup Meetings

Browse by Journal and Year


Lookup Conference Papers

Close Browse Journals / Lookup Meetings

Article Tools



Determination of the role of subject experience in the development of accurate color difference formulas is of potentially critical concern. As part of a larger multivariable experiment investigating the minimum inter- and intra-subject variability possible among a set of subjects, a study was conducted to compare the performance of 25 novice versus 25 expert visual assessors for a set of 27 pairs of colored textile samples using a controlled psychophysical method and several statistical techniques including t-test, ANOVA, and Standardized Residual Sum of Squares (STRESS) functions. Experts exhibited approximately 43% higher visual difference ratings than novice subjects when assessing sample pairs having small color differences. In addition, a statistically significant difference at the 95% confidence level was found between the judgments made by novice and expert assessors. According to the STRESS function, however, CMC(1:1) and CIEDE2000(1:1) color difference formulas do not show a significant difference in performance when the visual data from either group of subjects are compared.

© 2010 Optical Society of America

OCIS Codes
(330.1690) Vision, color, and visual optics : Color
(330.1720) Vision, color, and visual optics : Color vision
(330.5020) Vision, color, and visual optics : Perception psychology

ToC Category:
Vision, Color, and Visual Optics

Original Manuscript: October 27, 2009
Revised Manuscript: April 16, 2010
Manuscript Accepted: April 20, 2010
Published: May 27, 2010

Virtual Issues
Vol. 5, Iss. 10 Virtual Journal for Biomedical Optics

Renzo Shamey, Lina M. Cárdenas, David Hinks, and Roger Woodard, "Comparison of naïve and expert subjects in the assessment of small color differences," J. Opt. Soc. Am. A 27, 1482-1489 (2010)

Sort:  Author  |  Year  |  Journal  |  Reset  


  1. ISO, Textiles—ISO 105-J03 Tests for Colour Fastness—Part J03: Calculation of Colour Differences (ISO, 1995).
  2. ASTM, D2244-07 Standard Practice for Calculation of Color Tolerances and Color Differences from Instrumentally Measured Color Coordinates (ASTM International, 2007). [PubMed]
  3. SAE, SAE J1767: Instrumental Color Difference Measurements for Colorfastness of Automotive Interior Trim Materials (Society for Automotive Engineers, 1995). [PubMed]
  4. AATCC, Test Method 173-1998, “CMC: Calculation of small color differences for acceptability,” in AATCC Technical Manual, Vol. 80 (American Association of Textile Chemists and Colorists, 2005), pp. 311–315.
  5. CIE, Improvement to Industrial Colour-Difference Evaluation, CIE Publication No. 142 (CIE Central Bureau, 2001).
  6. M. Melgosa, R. Huertas, and R. Berns, “Performance of recent advanced color-difference formulas using the standardized residual sum of squares index,” J. Opt. Soc. Am. A 25, 1828–1834 (2008). [CrossRef]
  7. J. R. Aspland and P. Shanbhag, “Comparison of color-difference equations for textiles: CMC(2:1) and CIEDE2000,” AATCC Rev. 4, 26–30 (2004).
  8. J. Gay and R. Hirschler, “Field trials for CIEDE2000—correlation of visual and instrumental pass/fail decisions in industry,” presented at the 25th session of the CIE, San Diego, California, June 25–July 2, 2003.
  9. K. Noor, D. Hinks, A. Laidlaw, G. Treadaway, and R. Harold, “Comparison of the performance of CIEDE2000 and DECMC,” presented at AATCC International Conference and Exhibition, Greenville, South Carolina, September 10–12, 2003.
  10. G. M. Gibert, J. M. Daga, E. J. Gilabert, and J. Valldeperas, “Evaluation of color difference formulae,” Coloration Technol. 121, 147–151 (2005). [CrossRef]
  11. S. G. Lee, “Assessment of metrics in color spaces,” Master thesis (North Carolina State University, 2007).
  12. S. G. Lee, R. Shamey, D. Hinks, and W. Jasper, “Development of a comprehensive visual dataset based on a CIE blue color center: assessment of color difference formulae using various statistical methods,” Color Res. Appl. DOI:10.1002/col.20549 (2009).
  13. AATCC, Evaluation Procedure 1-1992, “Gray scale for color change,” in AATCC Technical Manual, Vol. 80 (AATCC, 2005), pp. 377–378.
  14. AATCC, Evaluation Procedure 9-2002, “Visual assessment of color difference of textiles,” in AATCC Technical Manual, Vol. 82 (AATCC, 2007), pp. 394–396.
  15. ASTM Committee E12, “D 2616-95 Standard test method for evaluation of visual color difference with a gray scale,” in ASTM Standards on Color Appearance Measurement, 7th ed. (ASTM International, 2004), pp. 123–125.
  16. J. Neitz, Manual: Neitz Test of Color Vision (Western Psychological Services, 2001).
  17. S. S. Guan and M. R. Luo, “Investigation of parametric effects using small colour differences,” Color Res. Appl. 24, 331–343 (1999). [CrossRef]
  18. “Statcrunch Vers. 4.0,” http://www.statcrunch.com.
  19. N. R. Farnum, Modern Statistical Quality Control and Improvement (Wadsworth, 1994).
  20. R. Lyman Ott and M. Longnecker, An Introduction to Statistical Methods and Data Analysis (Duxbury, 2001).
  21. L. M. Cárdenas, “Evaluation of variability in the visual assessment of small color differences,” Ph.D. thesis (North Carolina State University, 2009).
  22. P. A. García, R. Huertas, M. Melgosa, and G. Cui, “Measurement of the relationship between perceived and computed color differences,” J. Opt. Soc. Am. A 24, 1823–1829 (2007). [CrossRef]
  23. J. B. Kruskal, “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis,” Psychometrika 29, 1–27 (1964). [CrossRef]
  24. E. D. Montag and D. C. Wilber, “A comparison of constant stimuli and gray-scale methods of color difference scaling,” Color Res. Appl. 28, 36–44 (2003). [CrossRef]

Cited By

Alert me when this paper is cited

OSA is able to provide readers links to articles that cite this paper by participating in CrossRef's Cited-By Linking service. CrossRef includes content from more than 3000 publishers and societies. In addition to listing OSA journal articles that cite this paper, citing articles from other participating publishers will also be listed.


Fig. 1 Fig. 2 Fig. 3
Fig. 4 Fig. 5

« Previous Article  |  Next Article »

OSA is a member of CrossRef.

CrossCheck Deposited