Results from different authors showed deviations of radial orientation in the plane (tilt) for the major axes of chromaticity-discrimination ellipses centered around the International Commission on Illumination (CIE) red color center [Color Res. Appl. 3, 149 (1978)], which are not considered by most of the current advanced color-difference formulas (e.g., CIEDE2000). We performed a visual experiment using red printed samples in order to test the influence of the separation between samples (gap) on the mentioned tilt. Our results confirm a counterclockwise tilt of fitted ellipses with a magnitude of approximately 36° for samples with no separation, which is similar to that detected by other authors, and a reduction of the mentioned tilt owing to the separation of the samples. We detected a tilt of approximately 22° for samples with a black gap of 0.5 mm and a tilt of approximately 25° for samples with a white gap of 3 mm. Notably, the uncertainty of previous values given by the corresponding credibility intervals of 95% posterior probability is approximately of the mean values. Finally, we study the performance of the most widely used color-difference formulas in the graphic arts sector using our current experimental results, and conclude that the performance of the CAM02-SCD and CAM02-UCS color-difference formulas is significantly better than that of the CIEDE2000 formula.
© 2019 Optical Society of America
The International Commission on Illumination (CIE) has requested new reliable experimental data in order to improve color-difference formulas for industrial applications . CIE has also emphasized the relevance of viewing conditions (parametric effects) in color-difference evaluation . Some discrepancies have been observed in previous literature regarding a potential tilt (deviation with respect to radial orientation) for chromaticity-discrimination ellipses in the red area. The reason for such an effect is unknown, but it can be attributed to different factors, such as the separation between samples (gap), psychometric method, fitting methods, and material of the samples.
In the graphic arts industry, there is a particular interest in color discrimination under the usual viewing conditions.
This paper reports the results of a new experiment using printed samples in the red zone of CIELAB color space (CIE red color center ). Our main goal is to test the relationship between the tilt and size of the chromaticity-discrimination ellipses and the separation (gap) of the samples. Additionally, the performances of some recently proposed color-difference formulas are analyzed with the experimental data obtained.
Most of the current color-difference formulas (CIELAB, CIE94, CIEDE2000, CMC, CAM02-SCD, CAM02-UCS, and CAM02-LCD) have been developed from datasets published by different authors since the last half of the past century.
The adoption of new formulas or modification of the existing ones has been attempted to improve the correlation between metric differences, computed by means of a color-difference formula, and visual sensation of difference perceived by observers.
The color-difference formulas CIE94 and CMC represented an improvement with respect to CIELAB by considering the different sensibilities of the human vision system to the changes in lightness, chrome, and hue, which are considered by applying different weights in the formulas to the differences , .
CIEDE2000 was adopted by CIE/ISO  with the purpose, among others, of improving the behavior of CIE94 (the previously recommended color-difference formula by the CIE) in the area of blues and grays. One of the improvements was based on the introduction of an interactive term between chroma and hue differences to allow the rotation of the discrimination ellipses in the blue region because experimental datasets show that the major axis of the ellipses does not point toward the origin.
Luo et al.  described the development of the CIEDE2000 formula in detail. It was based on four experimental datasets: BFD-P (Luo and Rigg ), Leeds (Kim and Nobbs ), RIT-DuPont (Alman-Berns et al. [8,9]), and Witt . From these datasets (generated mostly between 1980 and 2000), the ellipses in the red zone maintain an orientation, which responds to the pattern of the color-difference formulas CIE94 and CMC, with the major axis pointing toward the origin of CIELAB color space.
Here, we define tilt () as the angular difference between the hue angle () of the color center and the angle of the major axis of the discrimination ellipses with respect to , in this order. Most authors have reported counterclockwise tilts for ellipses in the saturated red zone. Figure 1(a) shows the chromaticity-discrimination ellipses at the red zone obtained experimentally from the datasets BFD-P and RIT-DuPont compared with the corresponding ellipses obtained from the CIEDE2000 formula. The consistency between these two groups is evident, except for some ellipses far from the origin.
Table 1 summarizes the characteristics of the datasets used for the development of the CIEDE2000 formula. The last two columns indicate the CIELAB coordinates of the color centers closest to the CIE red color center (, , ) and the corresponding tilt (). The tilts in Table 1 for BFD-P and RIT-DuPont have been computed from Melgosa et al. [11,12], and those corresponding to the Witt  and Leeds  datasets have been calculated from the data collected in .
As shown in Table 1, the tilts of the ellipses near the CIE red color center are very small, except that of the Leeds dataset. However, the mentioned tilts at the CIE red center contrast with those reported by other authors [14–19], as shown in Fig. 1(b) and Table 2, and by Melgosa et al.  around the red–orange region.
It can be observed that most of the differences between Tables 1 and 2 are related to the psychometric method and the material of the samples. Most of the datasets of Table 1 have been collected from painted or textile samples using the gray scale method, whereas those from Table 2 have been collected from CRT or printed samples using the threshold method.
Wen  proposed an alternative color-difference formula to CIEDE2000 that considers the orientation pattern of MacAdam  chromaticity-discrimination ellipses. Wen concluded in his paper that CIEDE2000 is not suitable for predicting color differences at threshold because most of the datasets used for the development of the formula were obtained using the gray scale method or the constant stimuli method.
Huang et al.  investigated the performance of color-difference formulas for predicting threshold color differences surrounding 17 color centers recommended by the CIE . After an analysis using the STRESS index , they concluded that most of the current color-difference formulas performed similar to each other and outperformed CIELAB for small color differences.
Thus, the question of whether the differences of tilt and size of chromaticity-discrimination ellipses at the CIE red color center could be due to parametric effects caused by the psychometric method used (threshold, gray scale, or pair comparison) or by the separation between samples. As shown in Table 2, the tilt is much higher when samples without a contact hairline are displayed on a CRT.
Regarding the influence of the method on the tilt of the ellipses, several studies [24,25] have been conducted to compare the gray scale method with the method of constant stimuli, and there is no clear conclusion. Some authors  concluded that both methods are equivalent, whereas others  showed that the constant stimuli method provided a tolerance threshold approximately twice that obtained using the gray scale method. Given the high fiducial limits reported by these authors, a clear conclusion could not be established. In , the authors indicate that, at the CIE red center, the threshold and gray scale methods provide similar results, given that they observed a difference of 12.8 STRESS units. Additionally, if we compare Table 6 of  and Table VII of , the discrimination ellipses differ by a factor size of 0.9 and a tilt of 9° (difference between the tilt obtained using the gray scale method and that obtained using the threshold method), which can be considered small, given the usual variability of observers. Thus, taking into account the considerations made by the authors at the end of Section E of , it does not seem evident that the threshold method would be the main cause of the tilt detected at the CIE red center by the authors shown in Table 2.
Table 3 summarizes the results obtained by different authors regarding the effect of the separation of the samples (Factor column) on the size ( column) and tilt ( column) of the discrimination ellipses. More specifically, represents the average value of the ratio between the visual differences under the factor conditions to the visual differences under the reference conditions. Similarly, is the ratio factor between the volumes or areas of the ellipsoids or ellipses of discrimination under factor conditions to reference conditions. The smaller the effect of the factor, the closer to 1 and are. Finally, , , and are the ratios between the principal axes of the discrimination ellipsoids or ellipses under factor conditions to reference conditions.
As shown in the second row of Table 3, the effect determined by Strocka et al.  by observing pairs with a 1-mm-thick black line separation generates a deviation coefficient of approximately 7% with respect to the reference condition. Therefore, the authors concluded that the separation line does not generate significant effects in terms of the evaluation of the color differences with respect to the reference conditions. Witt  analyzed the effect of a gap of 3 mm between painted samples using the threshold method. The corresponding coefficients in Table 3 are calculated from the results reported in Table V of  considering the reference condition in this case to be the one corresponding to a lightness . As shown in Table 3, the tilt under the factor conditions (3 mm gap), and the tilt under the reference condition (dark surrounding) are different from the ones reported by other authors and even from the tilt reported by the author himself in  for the reference conditions, as shown in Table 1. The size factor shown in Table 3, which indicates that the ellipsoid of chromatic discrimination determined with a 3 mm gap is approximately twice as large as that corresponding to the reference conditions, is very high compared with that in the other studies shown in the same table.
However, the last four rows of Table 3 contradict the hypothesis defended by some authors by means of which an increase in the uncertainty, represented to a certain extent by an increase in the visual differences (), should necessarily be accompanied by an increase in the size of the ellipse or the ellipsoid of chromatic discrimination (). It can be observed that, as deduced from the data reported by Xin et al.  for painted samples, the effect of a 3” gap corresponds to and , whereas from the data reported by Cui et al. , for samples displayed on a screen monitor, the effect of a gap of 2” corresponds to and , which contradicts the mentioned hypothesis. Thus, an increase in uncertainty to perceive a color difference between a sample and a reference by observers need not necessarily be associated with an increase in the size of the chromaticity-discrimination ellipse.
It can be deduced from Table 3 that most of the parametric effects studied correspond to painted, textile, or CRT-displayed samples but not to printed samples on paper (very important for the graphics arts industry). Hence, in this paper, we will study the possible parametric effects owing to the separation of the samples on the evaluation of chromaticity differences at the CIE red color center [3,22], using the threshold method for printed samples on a white support.
Color samples were printed on a semigloss coated paper with an Epson SC 7000 inkjet plotter, equipped with 12 ink cartridges. Samples around the color center were printed in an attempt to follow an ideal pattern (similar to that used by Alman et al. ) of CIELAB color differences in eight vector directions angularly separated by 45° from 0 to 360° in the plane, and 10 pairs of samples in each direction with an increasing color difference of 0.4 units from 0 to 4, following an algorithm developed in Brusola et al. .
Three sets of color samples pairs were printed. The first set was printed with no separation between samples, the second was printed with samples separated by a 0.5-mm-thick black line, and the third was printed with a 3 mm white gap between samples (the same as that used by Witt ). The thickness of the black line was selected to lie between 1 mm proposed by Strocka et al.  and 0.35 mm (1 pixel displayed at 72 dpi) proposed by Cui et al. . Therefore, for each set, pairs of samples were printed.
Samples were printed with a size of , subtending an angle of 6° (according to the reference conditions for the CIE generalized color-difference equations ) to an observation distance of 40 cm, and were measured using a X-Rite I1 spectrophotometer. The CIELAB values were determined for D65 illuminant and 10° observer. The actual distribution of the measured color differences for pairs of printed samples with no separation is shown in Fig. 2. Similar results were obtained for the other two sets. Figure 2 shows the distribution of measured pairs of samples compared with the fitted values in eight vector directions obtained via linear regression passing through the origin.
The samples were presented to the observers in a Verivide viewing cabinet equipped with a Philips MASTER TL-D 90 Graphica 18 W/965 lamp, which has a correlated color temperature of 6500 K and color rendering index . The illuminance level at the test position was approximately 1000 lux. The background of the viewer corresponds to a neutral gray of coordinates .
Every sample was presented to a panel of 10–18 observers with normal color vision according to the Farnsworth–Munsell 100 Hue Test. The observers had to judge whether they could perceive a color difference between each pair of samples (threshold method). The observers repeated the judgments 4–10 times each, randomly in different sessions. Sets of pairs separated by a white line or black line were assessed 100 times, and the sets of pairs in contact were assessed 69 times.
To verify the performance of the most widely used color-difference formulas in the graphic arts sector with the data obtained, we have computed STRESS values and have studied the performance of color-difference formulas selected as indicated in [16,23].
The hypothesis test described in  is applied to the differences of quality of the adjustment provided by two different formulas of color difference with respect to the experimental data. We set the null hypothesis, , such that there is no significant difference between color-difference formulas A and B in terms of the quality of the fit provided, and the alternative hypothesis, , such that there are significant differences. Hence, if is true, the quotient, , calculated using Eq. (1) will be distributed according to a distribution with degrees of freedom and , where is the number of color pairs in the dataset.
Finally, we determine the tolerance that minimizes the number of wrong classification decisions with respect to a color-difference formula, according to Berns , which in turn allows the determination of the minimum percentage of wrong classification decisions with each formula selected.
The parameters of chromaticity-discrimination ellipses were determined using the logistic psychometric curve and observed fail frequency (number of times observers perceived a difference of color between the standard and the sample) using the Bayesian method described in . Table 4 shows the parameters of the chromaticity ellipses obtained for the three situations under study using the cited Bayesian procedure with the actual measured color differences without the linear fitting shown in Fig. 2.
As shown in Table 4, pair separation generates a reduction of the tilt between 11° and 14°. The 95% credibility intervals of the orientation of the ellipses are for samples in contact and separated with a gap of 3 mm. However, the credibility interval for samples separated by a black line is slightly lower (). Nevertheless, given the magnitude of the credibility intervals, we can conclude that the separation induces some kind of tilt reduction, but we cannot conclude that tilt reduction increases with the gap. The values of (standard deviation of the psychometric curve) shown in Table 4 indicate that the zone of uncertainty of the observers, in which they are doubtful about whether they can perceive color differences, is greater for the samples separated with a gap (black or white) and maximum for pairs with a 3-mm-thick white gap. However, as in some cases shown in Table 3, seldom increases with a 3 mm gap and even decreases with 0.5 mm gap, which indicates that 50% probability chromaticity-discrimination ellipse need not necessarily increase in size with the separation of the samples, as in  and , where decreased with the gap.
Table 5 shows the STRESS  values obtained when applying the CIELAB, CIE94, CIEDE2000, CMC, CAM02-SCD, CAM02-UCS, and CAM02-LCD color-difference formulas to the datasets of this work. We have chosen these color-difference formulas because they are the most widely used formulas in the graphics arts industrial sector. The parameters used for CAM02-UCS and CAM02-LCD color-difference formulas correspond to the following default values: adapting luminance, ; relative CIE Tristimulus of the background, (mid gray background), surround condition, (average), although the calculated values hardly vary for and (white background).
It can be observed that the values of STRESS when applying the formulas CAM02-SCD and CAM02-UCS are very similar (Table 5), which indicates that these formulas fit very well with the datasets obtained in this work. The results of application of the hypothesis test mentioned in Section 3 are shown in Tables 6–8.
Tables 6–8 show that, in general, there are no significant differences with respect to the quality of fit between most of the color-difference formulas used for the three datasets, except in the cases of the CIEDE2000 formula with respect to CAM02-SCD and CAM02-UCS formulas, in which there are significant differences in favor of the latter two formulas.
Thus, it can be concluded that the color-difference formulas CAM02-SCD and CAM02-UCS are the best ones with respect to the experimental data of this work.
A summary of the application of the procedure to determine the tolerances, with respect to the color-difference formulas evaluated, which minimized the number of wrong classification decisions, according to the procedure described by Berns , is shown in Table 9. As shown in Table 9, CAM02-SCD and CAM02-UCS formulas again produce the best results, with the lowest classification error rate. It can also be inferred from Table 9 that the optimal tolerance for minimizing the classification error should be 1 unit for the CIELAB color-difference formula and approximately 0.5 units for the remaining color-difference formulas. This seems to indicate that the acceptance limit should be approximately 0.5 units when attempting to make color comparisons at threshold with color-difference formulas other than CIELAB and 1 unit when using CIELAB. Moreover, this result seems to suggest that the tolerance of 0.5 could be the constant factor to add up when computing values from the frequency data in threshold experiments after applying the logit or probit transformation.
Finally, another interesting point that may be considered is the relationship between visual mechanisms and chromaticity-discrimination ellipses for samples with and without separation. In a recent paper, a specific experimental design was used to investigate the possible relationship between the separation of the samples and the “directions of color differences” in CIELAB color space . These authors reported that the effect of separation of samples for “tritan pairs” (two samples in which the difference between them is only due to the response of cones) was lower than for “deutan pairs” (pairs in which the ratio of responses of to cones signals is different for the two samples), which could be attributed to the negligible contribution of signal difference in producing a naturally occurring border between two abutting samples. This kind of perceptual anisotropy in CIELAB color space may be considered consistent with our current results, and the differences between tritan and deutan mechanisms may be a potential explanation of tilt in chromaticity-discrimination ellipses. Specifically, if separation of samples increases visual differences in the deutan direction ( axis) more than in the tritan direction ( axis), as indicated in Fig. 5 of Ref. , the corresponding result should be a decrease in the orientation and tilt of the ellipses, which is in agreement with values shown in Table 4. Generally speaking, the relationship between confusion lines and chromaticity-discrimination ellipses is not new in the literature. For example, Le Grand  already pointed out that the major axes of MacAdam’s ellipses  are almost parallel to the axis corresponding to the blue fundamental response, and some relationship between tilt of ellipses in the blue region and tritan confusion lines has been also reported in previous literature .
Three datasets have been generated to analyze parametric factors for printed samples around the CIE red color center using the threshold method. From the analysis, it is concluded that the chromaticity-discrimination ellipses have an average tilt of 36° for samples with no separation, a tilt of approximately 22° for samples with a black gap of 0.5 mm, and a tilt of approximately 25° for samples with a white gap of 3 mm. The results for samples with no separation are consistent with those reported by Huang et al.  and are approximately the average of the results reported by Melgosa et al.  for the red–orange region.
The results also show that the parametric factors related to the separation between samples have a slight effect on the reduction of the tilt (approximately 10°) with respect to the tilt of the chromaticity-discrimination ellipse for samples with no separation.
A minimum wrong classification error rate not greater than 10% is generated by most of the color-difference formulas evaluated in this work. Nevertheless, CAM02-SCD and CAM02-UCS provide the best values, approximately 5%.
The results show that all color-difference formulas performed similarly, but CAM02-SCD and CAM02-UCS performed significantly better than CIEDE2000.
In summary, as it was requested by CIE , the authors consider that future studies should be performed in other color centers near the red zone in order to finally make a possible decision of improving CIEDE2000 or recommend the use of other color-difference formulas such as CAM02-SCD and CAM02-UCS, when evaluating color differences at threshold in the red zone.
1. M. Melgosa, “Request for existing experimental datasets on color differences,” Color Res. Appl. 32, 159 (2007). [CrossRef]
2. CIE, “Parametric effects in colour-difference evaluation,” CIE 101-1993 (CIE Central Bureau, 1993).
3. A. R. Robertson, “CIE guidelines for coordinated research on colour-difference evaluation,” Color Res. Appl. 3, 149–151 (1978).
4. ISO/CIE, “Colorimetry—Part 6: CIEDE2000 colour-difference formula,” ISO/CIE 11664-6:2014 (2014).
5. M. R. Luo, G. Cui, and B. Rigg, “The development of the CIE 2000 colour difference formula,” Color Res. Appl. 26, 340–350 (2001). [CrossRef]
6. M. R. Luo and B. Rigg, “Chromaticity-discrimination ellipses for surface colours,” Color Res. Appl. 11, 25–42 (1986). [CrossRef]
7. D. H. Kim and J. H. Nobbs, “New weighting functions for the weighted CIELAB colour difference formula,” in 8th International Colour Association, Kyoto, Japan, 1997, Vol. 1, pp. 446–449.
8. D. H. Alman, R. S. Berns, G. D. Snyder, and W. A. Larsen, “Performance testing of color-difference metrics using a color tolerance dataset,” Color Res. Appl. 14, 139–151 (1989). [CrossRef]
9. R. S. Berns, D. H. Alman, L. Reniff, G. D. Snyder, and M. R. Balonon-Rosen, “Visual determination of suprathreshold color-difference tolerances using probit analysis,” Color Res. Appl. 16, 297–316 (1991). [CrossRef]
10. K. Witt, “Geometric relations between scales of small colour differences,” Color Res. Appl. 24, 78–92 (1999). [CrossRef]
11. M. Melgosa, E. Hita, J. Romero, and L. J. Del Barco, “Color-discrimination thresholds translated from the CIE (x, y, Y) space to the CIE 1976 (L*, a*, b*),” Color Res. Appl. 19, 10–18 (1994). [CrossRef]
12. M. Melgosa, E. Hita, A. J. Poza, D. H. Alman, and R. S. Berns, “Suprathreshold color-difference ellipsoids for surface colors,” Color Res. Appl. 22, 148–155 (1997). [CrossRef]
14. T. Indow, A. R. Robertson, M. Grunau, and G. H. Fielder, “Discrimination ellipsoids of aperture and simulated surface colors by matching and paired comparison,” Color Res. Appl. 17, 6–23 (1992). [CrossRef]
15. H. Xu and H. Yaguchi, “Visual evaluation at scale of threshold to suprathreshold color difference,” Color Res. Appl. 30, 198–208 (2005). [CrossRef]
16. M. Huang, H. Liu, G. Cui, M. R. Luo, and M. Melgosa, “Evaluation of threshold color differences using printed samples,” J. Opt. Soc. Am. A 29, 883–891 (2012). [CrossRef]
17. S. Wen, “A color difference metric based on the chromaticity discrimination ellipses,” Opt. Express 20, 26441–26447 (2012). [CrossRef]
18. M. Huang, H. Liu, G. Cui, and M. R. Luo, “Testing uniform color spaces and colour-difference formulae using printed samples,” Color Res. Appl. 37, 326–335 (2012). [CrossRef]
19. R. M. Rich and F. W. Billmeyer, “Method for deriving color-difference-perceptibility ellipses for surface-color samples,” J. Opt. Soc. Am. 65, 956–959 (1975). [CrossRef]
20. M. Melgosa, R. Huertas, A. Yebra, and E. Hita, “A comparison between the rotation terms proposed by recent CIELAB-based color-difference formulas,” in 1st European Conference on Colour in Graphics, Image and Vision (CGIV) (2002), pp. 136–139.
21. D. L. MacAdam, “Visual sensitivities to color differences in daylight,” J. Opt. Soc. Am. 32, 247–274 (1942). [CrossRef]
22. T. Maier, “CIE guidelines for coordinated future work on industrial color-difference evaluation,” Color Res. Appl. 20, 399–403 (1995). [CrossRef]
23. P. A. García, R. Huertas, M. Melgosa, and G. Cui, “Measurement of the relationship between perceived and computed color differences,” J. Opt. Soc. Am. A 24, 1823–1829 (2007). [CrossRef]
24. S. S. Guan and M. R. Luo, “Investigation of parametric effects using small colour differences,” Color Res. Appl. 24, 331–343 (1999). [CrossRef]
25. E. D. Montag and D. C. Wilber, “A comparison of constant stimuli and gray-scale methods of color difference scaling,” Color Res. Appl. 28, 36–44 (2003). [CrossRef]
26. D. Strocka, A. Brockes, and W. Paffhausen, “Influence of experimental parameters on the evaluation of color-difference ellipsoids,” Color Res. Appl. 8, 169–175 (1983). [CrossRef]
27. K. Witt, “Parametric effects on surface color-difference evaluation at threshold,” Color Res. Appl. 15, 189–199 (1990). [CrossRef]
28. J. H. Xin, C. C. Lam, and M. R. Luo, “Investigation of parametric effects using medium colour-difference pairs,” Color Res. Appl. 26, 376–383 (2001). [CrossRef]
29. G. Cui, M. R. Luo, B. Rigg, and W. Li, “Colour-difference evaluation using CRT colours. Part II: parametric effects,” Color Res. Appl. 26, 403–412 (2001). [CrossRef]
30. F. Brusola, I. Tortajada, B. Jordá, I. Lengua, and L. Dunai, “Método para la reproducción precisa de muestras de color impresas a partir de los datos de caracterización necesarios para la generación de perfiles de color ICC,” in 11th Congreso Nacional de Color, Ourense, Spain, 2016, pp. 109–112.
31. R. S. Berns, “Deriving instrumental tolerances from pass-fail and colorimetric data,” Color Res. Appl. 21, 459–472 (1996). [CrossRef]
32. F. Brusola, I. Tortajada, I. Lengua, B. Jordá, and G. Peris, “Bayesian approach to color-difference models based on threshold and constant-stimuli methods,” Opt. Express 23, 15290–15309 (2015). [CrossRef]
33. H. Saeedi and S. G. Kandi, “How anisotropy of CIELAB color space affects the separation effect: an experimental study,” J. Opt. Soc. Am. A 36, 51–60 (2019). [CrossRef]
34. Y. Le Grand, “Les seuils différentiels de couleurs dans la théorie de Young,” Revue d’Optique 28, 261–278 (1949) [R. M. Boynton and K. Knoblauch, “Color difference thresholds in Young’s theory,” Color Res. Appl. 19, 296–309 (1994)].
35. A. Yebra, R. Huertas, M. M. Pérez, and M. Melgosa, “On the relationship between tilt of a*b* tolerance ellipses in blue region and tritanopic confusion lines,” Color Res. Appl. 27, 180–184 (2002). [CrossRef]