Abstract

Determination of the role of subject experience in the development of accurate color difference formulas is of potentially critical concern. As part of a larger multivariable experiment investigating the minimum inter- and intra-subject variability possible among a set of subjects, a study was conducted to compare the performance of 25 novice versus 25 expert visual assessors for a set of 27 pairs of colored textile samples using a controlled psychophysical method and several statistical techniques including t-test, ANOVA, and Standardized Residual Sum of Squares (STRESS) functions. Experts exhibited approximately 43% higher visual difference ratings than novice subjects when assessing sample pairs having small color differences. In addition, a statistically significant difference at the 95% confidence level was found between the judgments made by novice and expert assessors. According to the STRESS function, however, CMC(1:1) and CIEDE2000(1:1) color difference formulas do not show a significant difference in performance when the visual data from either group of subjects are compared.

© 2010 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. ISO, Textiles—ISO 105-J03 Tests for Colour Fastness—Part J03: Calculation of Colour Differences (ISO, 1995).
  2. ASTM, D2244-07 Standard Practice for Calculation of Color Tolerances and Color Differences from Instrumentally Measured Color Coordinates (ASTM International, 2007).
    [PubMed]
  3. SAE, SAE J1767: Instrumental Color Difference Measurements for Colorfastness of Automotive Interior Trim Materials (Society for Automotive Engineers, 1995).
    [PubMed]
  4. AATCC, Test Method 173-1998, “CMC: Calculation of small color differences for acceptability,” in AATCC Technical Manual, Vol. 80 (American Association of Textile Chemists and Colorists, 2005), pp. 311–315.
  5. CIE, Improvement to Industrial Colour-Difference Evaluation, CIE Publication No. 142 (CIE Central Bureau, 2001).
  6. M. Melgosa, R. Huertas, and R. Berns, “Performance of recent advanced color-difference formulas using the standardized residual sum of squares index,” J. Opt. Soc. Am. A 25, 1828–1834 (2008).
    [CrossRef]
  7. J. R. Aspland and P. Shanbhag, “Comparison of color-difference equations for textiles: CMC(2:1) and CIEDE2000,” AATCC Rev. 4, 26–30 (2004).
  8. J. Gay and R. Hirschler, “Field trials for CIEDE2000—correlation of visual and instrumental pass/fail decisions in industry,” presented at the 25th session of the CIE, San Diego, California, June 25–July 2, 2003.
  9. K. Noor, D. Hinks, A. Laidlaw, G. Treadaway, and R. Harold, “Comparison of the performance of CIEDE2000 and DECMC,” presented at AATCC International Conference and Exhibition, Greenville, South Carolina, September 10–12, 2003.
  10. G. M. Gibert, J. M. Daga, E. J. Gilabert, and J. Valldeperas, “Evaluation of color difference formulae,” Coloration Technol. 121, 147–151 (2005).
    [CrossRef]
  11. S. G. Lee, “Assessment of metrics in color spaces,” Master thesis (North Carolina State University, 2007).
  12. S. G. Lee, R. Shamey, D. Hinks, and W. Jasper, “Development of a comprehensive visual dataset based on a CIE blue color center: assessment of color difference formulae using various statistical methods,” Color Res. Appl. DOI:10.1002/col.20549 (2009).
  13. AATCC, Evaluation Procedure 1-1992, “Gray scale for color change,” in AATCC Technical Manual, Vol. 80 (AATCC, 2005), pp. 377–378.
  14. AATCC, Evaluation Procedure 9-2002, “Visual assessment of color difference of textiles,” in AATCC Technical Manual, Vol. 82 (AATCC, 2007), pp. 394–396.
  15. ASTM Committee E12, “D 2616-95 Standard test method for evaluation of visual color difference with a gray scale,” in ASTM Standards on Color Appearance Measurement, 7th ed. (ASTM International, 2004), pp. 123–125.
  16. J. Neitz, Manual: Neitz Test of Color Vision (Western Psychological Services, 2001).
  17. S. S. Guan and M. R. Luo, “Investigation of parametric effects using small colour differences,” Color Res. Appl. 24, 331–343 (1999).
    [CrossRef]
  18. “Statcrunch Vers. 4.0,” http://www.statcrunch.com.
  19. N. R. Farnum, Modern Statistical Quality Control and Improvement (Wadsworth, 1994).
  20. R. Lyman Ott and M. Longnecker, An Introduction to Statistical Methods and Data Analysis (Duxbury, 2001).
  21. L. M. Cárdenas, “Evaluation of variability in the visual assessment of small color differences,” Ph.D. thesis (North Carolina State University, 2009).
  22. P. A. García, R. Huertas, M. Melgosa, and G. Cui, “Measurement of the relationship between perceived and computed color differences,” J. Opt. Soc. Am. A 24, 1823–1829 (2007).
    [CrossRef]
  23. J. B. Kruskal, “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis,” Psychometrika 29, 1–27 (1964).
    [CrossRef]
  24. E. D. Montag and D. C. Wilber, “A comparison of constant stimuli and gray-scale methods of color difference scaling,” Color Res. Appl. 28, 36–44 (2003).
    [CrossRef]

2009 (2)

S. G. Lee, R. Shamey, D. Hinks, and W. Jasper, “Development of a comprehensive visual dataset based on a CIE blue color center: assessment of color difference formulae using various statistical methods,” Color Res. Appl. DOI:10.1002/col.20549 (2009).

L. M. Cárdenas, “Evaluation of variability in the visual assessment of small color differences,” Ph.D. thesis (North Carolina State University, 2009).

2008 (1)

2007 (4)

P. A. García, R. Huertas, M. Melgosa, and G. Cui, “Measurement of the relationship between perceived and computed color differences,” J. Opt. Soc. Am. A 24, 1823–1829 (2007).
[CrossRef]

ASTM, D2244-07 Standard Practice for Calculation of Color Tolerances and Color Differences from Instrumentally Measured Color Coordinates (ASTM International, 2007).
[PubMed]

S. G. Lee, “Assessment of metrics in color spaces,” Master thesis (North Carolina State University, 2007).

AATCC, Evaluation Procedure 9-2002, “Visual assessment of color difference of textiles,” in AATCC Technical Manual, Vol. 82 (AATCC, 2007), pp. 394–396.

2005 (3)

G. M. Gibert, J. M. Daga, E. J. Gilabert, and J. Valldeperas, “Evaluation of color difference formulae,” Coloration Technol. 121, 147–151 (2005).
[CrossRef]

AATCC, Test Method 173-1998, “CMC: Calculation of small color differences for acceptability,” in AATCC Technical Manual, Vol. 80 (American Association of Textile Chemists and Colorists, 2005), pp. 311–315.

AATCC, Evaluation Procedure 1-1992, “Gray scale for color change,” in AATCC Technical Manual, Vol. 80 (AATCC, 2005), pp. 377–378.

2004 (2)

J. R. Aspland and P. Shanbhag, “Comparison of color-difference equations for textiles: CMC(2:1) and CIEDE2000,” AATCC Rev. 4, 26–30 (2004).

ASTM Committee E12, “D 2616-95 Standard test method for evaluation of visual color difference with a gray scale,” in ASTM Standards on Color Appearance Measurement, 7th ed. (ASTM International, 2004), pp. 123–125.

2003 (3)

J. Gay and R. Hirschler, “Field trials for CIEDE2000—correlation of visual and instrumental pass/fail decisions in industry,” presented at the 25th session of the CIE, San Diego, California, June 25–July 2, 2003.

K. Noor, D. Hinks, A. Laidlaw, G. Treadaway, and R. Harold, “Comparison of the performance of CIEDE2000 and DECMC,” presented at AATCC International Conference and Exhibition, Greenville, South Carolina, September 10–12, 2003.

E. D. Montag and D. C. Wilber, “A comparison of constant stimuli and gray-scale methods of color difference scaling,” Color Res. Appl. 28, 36–44 (2003).
[CrossRef]

2001 (3)

R. Lyman Ott and M. Longnecker, An Introduction to Statistical Methods and Data Analysis (Duxbury, 2001).

J. Neitz, Manual: Neitz Test of Color Vision (Western Psychological Services, 2001).

CIE, Improvement to Industrial Colour-Difference Evaluation, CIE Publication No. 142 (CIE Central Bureau, 2001).

1999 (1)

S. S. Guan and M. R. Luo, “Investigation of parametric effects using small colour differences,” Color Res. Appl. 24, 331–343 (1999).
[CrossRef]

1995 (2)

SAE, SAE J1767: Instrumental Color Difference Measurements for Colorfastness of Automotive Interior Trim Materials (Society for Automotive Engineers, 1995).
[PubMed]

ISO, Textiles—ISO 105-J03 Tests for Colour Fastness—Part J03: Calculation of Colour Differences (ISO, 1995).

1994 (1)

N. R. Farnum, Modern Statistical Quality Control and Improvement (Wadsworth, 1994).

1964 (1)

J. B. Kruskal, “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis,” Psychometrika 29, 1–27 (1964).
[CrossRef]

Aspland, J. R.

J. R. Aspland and P. Shanbhag, “Comparison of color-difference equations for textiles: CMC(2:1) and CIEDE2000,” AATCC Rev. 4, 26–30 (2004).

Berns, R.

Cárdenas, L. M.

L. M. Cárdenas, “Evaluation of variability in the visual assessment of small color differences,” Ph.D. thesis (North Carolina State University, 2009).

Cui, G.

Daga, J. M.

G. M. Gibert, J. M. Daga, E. J. Gilabert, and J. Valldeperas, “Evaluation of color difference formulae,” Coloration Technol. 121, 147–151 (2005).
[CrossRef]

Farnum, N. R.

N. R. Farnum, Modern Statistical Quality Control and Improvement (Wadsworth, 1994).

García, P. A.

Gay, J.

J. Gay and R. Hirschler, “Field trials for CIEDE2000—correlation of visual and instrumental pass/fail decisions in industry,” presented at the 25th session of the CIE, San Diego, California, June 25–July 2, 2003.

Gibert, G. M.

G. M. Gibert, J. M. Daga, E. J. Gilabert, and J. Valldeperas, “Evaluation of color difference formulae,” Coloration Technol. 121, 147–151 (2005).
[CrossRef]

Gilabert, E. J.

G. M. Gibert, J. M. Daga, E. J. Gilabert, and J. Valldeperas, “Evaluation of color difference formulae,” Coloration Technol. 121, 147–151 (2005).
[CrossRef]

Guan, S. S.

S. S. Guan and M. R. Luo, “Investigation of parametric effects using small colour differences,” Color Res. Appl. 24, 331–343 (1999).
[CrossRef]

Harold, R.

K. Noor, D. Hinks, A. Laidlaw, G. Treadaway, and R. Harold, “Comparison of the performance of CIEDE2000 and DECMC,” presented at AATCC International Conference and Exhibition, Greenville, South Carolina, September 10–12, 2003.

Hinks, D.

S. G. Lee, R. Shamey, D. Hinks, and W. Jasper, “Development of a comprehensive visual dataset based on a CIE blue color center: assessment of color difference formulae using various statistical methods,” Color Res. Appl. DOI:10.1002/col.20549 (2009).

K. Noor, D. Hinks, A. Laidlaw, G. Treadaway, and R. Harold, “Comparison of the performance of CIEDE2000 and DECMC,” presented at AATCC International Conference and Exhibition, Greenville, South Carolina, September 10–12, 2003.

Hirschler, R.

J. Gay and R. Hirschler, “Field trials for CIEDE2000—correlation of visual and instrumental pass/fail decisions in industry,” presented at the 25th session of the CIE, San Diego, California, June 25–July 2, 2003.

Huertas, R.

Jasper, W.

S. G. Lee, R. Shamey, D. Hinks, and W. Jasper, “Development of a comprehensive visual dataset based on a CIE blue color center: assessment of color difference formulae using various statistical methods,” Color Res. Appl. DOI:10.1002/col.20549 (2009).

Kruskal, J. B.

J. B. Kruskal, “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis,” Psychometrika 29, 1–27 (1964).
[CrossRef]

Laidlaw, A.

K. Noor, D. Hinks, A. Laidlaw, G. Treadaway, and R. Harold, “Comparison of the performance of CIEDE2000 and DECMC,” presented at AATCC International Conference and Exhibition, Greenville, South Carolina, September 10–12, 2003.

Lee, S. G.

S. G. Lee, R. Shamey, D. Hinks, and W. Jasper, “Development of a comprehensive visual dataset based on a CIE blue color center: assessment of color difference formulae using various statistical methods,” Color Res. Appl. DOI:10.1002/col.20549 (2009).

S. G. Lee, “Assessment of metrics in color spaces,” Master thesis (North Carolina State University, 2007).

Longnecker, M.

R. Lyman Ott and M. Longnecker, An Introduction to Statistical Methods and Data Analysis (Duxbury, 2001).

Luo, M. R.

S. S. Guan and M. R. Luo, “Investigation of parametric effects using small colour differences,” Color Res. Appl. 24, 331–343 (1999).
[CrossRef]

Lyman Ott, R.

R. Lyman Ott and M. Longnecker, An Introduction to Statistical Methods and Data Analysis (Duxbury, 2001).

Melgosa, M.

Montag, E. D.

E. D. Montag and D. C. Wilber, “A comparison of constant stimuli and gray-scale methods of color difference scaling,” Color Res. Appl. 28, 36–44 (2003).
[CrossRef]

Neitz, J.

J. Neitz, Manual: Neitz Test of Color Vision (Western Psychological Services, 2001).

Noor, K.

K. Noor, D. Hinks, A. Laidlaw, G. Treadaway, and R. Harold, “Comparison of the performance of CIEDE2000 and DECMC,” presented at AATCC International Conference and Exhibition, Greenville, South Carolina, September 10–12, 2003.

Shamey, R.

S. G. Lee, R. Shamey, D. Hinks, and W. Jasper, “Development of a comprehensive visual dataset based on a CIE blue color center: assessment of color difference formulae using various statistical methods,” Color Res. Appl. DOI:10.1002/col.20549 (2009).

Shanbhag, P.

J. R. Aspland and P. Shanbhag, “Comparison of color-difference equations for textiles: CMC(2:1) and CIEDE2000,” AATCC Rev. 4, 26–30 (2004).

Treadaway, G.

K. Noor, D. Hinks, A. Laidlaw, G. Treadaway, and R. Harold, “Comparison of the performance of CIEDE2000 and DECMC,” presented at AATCC International Conference and Exhibition, Greenville, South Carolina, September 10–12, 2003.

Valldeperas, J.

G. M. Gibert, J. M. Daga, E. J. Gilabert, and J. Valldeperas, “Evaluation of color difference formulae,” Coloration Technol. 121, 147–151 (2005).
[CrossRef]

Wilber, D. C.

E. D. Montag and D. C. Wilber, “A comparison of constant stimuli and gray-scale methods of color difference scaling,” Color Res. Appl. 28, 36–44 (2003).
[CrossRef]

AATCC Rev. (1)

J. R. Aspland and P. Shanbhag, “Comparison of color-difference equations for textiles: CMC(2:1) and CIEDE2000,” AATCC Rev. 4, 26–30 (2004).

Color Res. Appl. (3)

S. G. Lee, R. Shamey, D. Hinks, and W. Jasper, “Development of a comprehensive visual dataset based on a CIE blue color center: assessment of color difference formulae using various statistical methods,” Color Res. Appl. DOI:10.1002/col.20549 (2009).

S. S. Guan and M. R. Luo, “Investigation of parametric effects using small colour differences,” Color Res. Appl. 24, 331–343 (1999).
[CrossRef]

E. D. Montag and D. C. Wilber, “A comparison of constant stimuli and gray-scale methods of color difference scaling,” Color Res. Appl. 28, 36–44 (2003).
[CrossRef]

Coloration Technol. (1)

G. M. Gibert, J. M. Daga, E. J. Gilabert, and J. Valldeperas, “Evaluation of color difference formulae,” Coloration Technol. 121, 147–151 (2005).
[CrossRef]

J. Opt. Soc. Am. A (2)

Psychometrika (1)

J. B. Kruskal, “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis,” Psychometrika 29, 1–27 (1964).
[CrossRef]

Other (16)

S. G. Lee, “Assessment of metrics in color spaces,” Master thesis (North Carolina State University, 2007).

J. Gay and R. Hirschler, “Field trials for CIEDE2000—correlation of visual and instrumental pass/fail decisions in industry,” presented at the 25th session of the CIE, San Diego, California, June 25–July 2, 2003.

K. Noor, D. Hinks, A. Laidlaw, G. Treadaway, and R. Harold, “Comparison of the performance of CIEDE2000 and DECMC,” presented at AATCC International Conference and Exhibition, Greenville, South Carolina, September 10–12, 2003.

AATCC, Evaluation Procedure 1-1992, “Gray scale for color change,” in AATCC Technical Manual, Vol. 80 (AATCC, 2005), pp. 377–378.

AATCC, Evaluation Procedure 9-2002, “Visual assessment of color difference of textiles,” in AATCC Technical Manual, Vol. 82 (AATCC, 2007), pp. 394–396.

ASTM Committee E12, “D 2616-95 Standard test method for evaluation of visual color difference with a gray scale,” in ASTM Standards on Color Appearance Measurement, 7th ed. (ASTM International, 2004), pp. 123–125.

J. Neitz, Manual: Neitz Test of Color Vision (Western Psychological Services, 2001).

“Statcrunch Vers. 4.0,” http://www.statcrunch.com.

N. R. Farnum, Modern Statistical Quality Control and Improvement (Wadsworth, 1994).

R. Lyman Ott and M. Longnecker, An Introduction to Statistical Methods and Data Analysis (Duxbury, 2001).

L. M. Cárdenas, “Evaluation of variability in the visual assessment of small color differences,” Ph.D. thesis (North Carolina State University, 2009).

ISO, Textiles—ISO 105-J03 Tests for Colour Fastness—Part J03: Calculation of Colour Differences (ISO, 1995).

ASTM, D2244-07 Standard Practice for Calculation of Color Tolerances and Color Differences from Instrumentally Measured Color Coordinates (ASTM International, 2007).
[PubMed]

SAE, SAE J1767: Instrumental Color Difference Measurements for Colorfastness of Automotive Interior Trim Materials (Society for Automotive Engineers, 1995).
[PubMed]

AATCC, Test Method 173-1998, “CMC: Calculation of small color differences for acceptability,” in AATCC Technical Manual, Vol. 80 (American Association of Textile Chemists and Colorists, 2005), pp. 311–315.

CIE, Improvement to Industrial Colour-Difference Evaluation, CIE Publication No. 142 (CIE Central Bureau, 2001).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1

Location of dyed samples on the CIE a * b * plane including the standards.

Fig. 2
Fig. 2

Experimental setup employed for visual assessment of color difference using an AATCC gray scale for color change.

Fig. 3
Fig. 3

Δ V for the assessments of each sample pair in each trial for novice and expert subjects, ranked in ascending Δ E * ab , 10 order.

Fig. 4
Fig. 4

Subject repeatability represented by the difference in STRESS repeatability for Trials 2 and 3 minus STRESS repeatability for Trials 1 and 2.

Fig. 5
Fig. 5

STRESS values for Δ E * ab , CMC(1:1), and CIEDE2000(1:1:1) for the mean novice subjects’ assessments in each trial and mean expert subject responses.

Tables (7)

Tables Icon

Table 7 Colorometric Data Pertaining to the 27 Sample Pairs

Tables Icon

Table 1 Summary of t-Test Statistics for Assessments Carried Out by Novice Assessors and Summary Statistics for Novice Subject versus Expert Subject Trials

Tables Icon

Table 2 Variance Components Estimates for Expert and Novice Subjects

Tables Icon

Table 3 Summary of Intra-Subject Variability Expressed by STRESS for Accuracy

Tables Icon

Table 4 Summary of STRESS for Repeatability of Assessments by Novice Subjects

Tables Icon

Table 5 STRESS Values for the Performance of Color Difference Models Based on Novice and Expert Visual Data

Tables Icon

Table 6 F Values between Different Equations for Each Set of Subjects Indicating the Significance of Difference between Models a

Equations (2)

Equations on this page are rendered with MathJax. Learn more.

Δ V = 0.180 G 3 + 2.421 G 2 12.312 G + 23.749 .
% = i = 1 I ( D V ̿ i ( experts ) D V ̿ i ( novice ) D V ̿ i ( novice ) ) × 100 I .

Metrics