Open Access
Add to BibSonomy
Abstract
A personalized image enhancement method is proposed to improve color vision in hereditary color vision deficiency (CVD). It is divided into two stages: evaluation of CVD and gamut mapping for image enhancement. These two separate stages are connected via a psychophysical experiment, through which the deficiency test result expressed using the C-index can be further transformed into a physical parameter, namely the wavelength shift of the cone fundamental. Experiments conducted by the color-deficient observers (CDOs) validated this proposed method, and it is emphasized that the proposed method is just serving as a template for image enhancement. A more advanced simulation model, a more accurate assessment method, or a more sophisticated gamut mapping algorithm can yield a better result.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Human color vision is typically trichromatic, with responses from three types of cone photoreceptors (L, M, and S) [1]. These three cones have distinct peak wavelengths. However, in commonly inherited color defects, one of the cone fundamentals is missing or altered, resulting in the absence of one receptor type (dichromacy) or a change in wavelength sensitivity of the altered cone, making it more comparable to the normal M or L cone (anomalous trichromacy).
CDOs are frequently overlooked when providing clear access to visual information on multimedia-enabled devices. According to large random population studies, the prevalence of deficiency among European Caucasians is around 8% in males and 0.4% in females, and between 4% and 6.5% in men of Chinese and Japanese ancestry [2]. It is also hypothesized that male prevalence increases after migrants with a higher frequency of deficiency enter the area and intermarry. Such a figure should not be neglected, and it is desirable if one can assist them in having easy and equal access to all types of color information.
Before using an image enhancing technique, it is critical to accurately diagnose the type and severity of a CDO. Generally, the assessment methods can be classified into three major types [3], i.e., generic test, computer-based test, and gene test. Generic test includes the traditional pseudoisochromatic-plate (PIP) test, the hue-arrangement test, and the color-matching test. PIP tests are the most commonly used to determine the presence of color vision deficiency, but they cannot be used to assess the severity of the deficiency. The hue-arrangement test can determine both the type and severity of CVD by assessing the color discrimination capability of CDOs using numerous colored caps with varying perceived hues. The color-match test, which includes the Rayleigh match, is widely regarded as the gold standard for determining the type and degree of color deficiency. Computer-based tests are growing increasingly prevalent in recent years, and they provide more accurate diagnosis results. The majority of these approaches rely on chromatic discrimination metrics. In comparison to the other procedures, the gene test works on a completely different concept. It does not have the subjects completing a time-consuming color task, but it does necessitate the use of professional equipment to extract and analyze their genetic components (DNA). All of these evaluation approaches will be briefly discussed.
The representative of PIP tests is the Ishihara test plate [4]. It is made up of several test objects, each of which contains letters, numbers, or geometric shapes enclosed by colored dots of varying sizes, embedding in a background differentiated of the same luminous reflectance with only color differences. CDOs will have difficulty to recognize some or all of the objects because the figures and their backgrounds are colored in easily confused colors selected along dichromatic confusion lines. CDOs are classified as mild, moderate, or severe based on the results of these tests. One thing should be noted is that, the Ishihara test identifies only red-green color vision deficiencies since no confusion colors for tritan are included. It provides a useful tool for testing the deficient type but cannot provide a precise degree of deficient severity of a CDO. One special type of PIP is the Hardy Rand-Rittler (HRR) test [5], which consists of a series of diagnostic plates with increasing color differences. It can detect not only the type, but also the severity.
One of the most popular hue-arrangement tests would be the Farnsworth Munsell 100 Hue Test (FM100) [6]. It consists of 85 colored caps divided into four panels: red to yellow-green (No. 84 to No. 22), yellow-green to green-blue (No.21 to No.43), green-blue to blue-red (No.42 to No.64), and blue-red to red (No.63 to No.85). Observers are asked to arrange these color patches in the order of their hues. Following each arrangement, an error score based on the placement of each color cap will be calculated. A higher score indicates poor performance and a more severe color deficiency. Although the error score can be used to demonstrate an observer's color discrimination ability, there is no clearly defined relationship between the error score and the severity of color deficiency. Its distribution of color caps does not follow the confusion lines, which is a great advantage of this strategy. This means that no assumptions were made before to the test (e.g., this subject was a protan CDO) making it appropriate to detect all types of CVD, even acquired ones. In addition, the test is challenging enough to distinguish between the severity levels of various CDOs. Color caps for FM100, however, are picked from Munsell samples, resulting in an unequal distribution of hue direction (see Fig. 1). Furthermore, there is a systematic variation of lightness for various color caps, particularly in the red zone [7]. Given their loss of red sensitivity, this provides a clue for protans.
Fig. 1. (a) Color distribution of the caps from FM100 in CIELAB color space and (b) in CAM16-UCS color space (right).
Computer-based tests are becoming more common as computer science and display technologies advance. The majority of them are based on chromatic discrimination detection and can provide exact diagnosis results that are equivalent to other methods. The Cambridge Color Test (CCT) [8] is probably the most extensively used tool of this type. This test uses the Landolt C stimulus, and individuals are asked to find the gap in the C stimulus. With a staircase procedure, the colors of the stimulus and its backgrounds vary in chromaticity along all three confusion axes (protan, deutan, and tritan). As a result, the chromatic sensitivities along each line can be established, as well as the type and severity. The main advantage of this technology is to allow for fine color control and easy randomization of presentation order, which can reduce systematic uncertainty to some amount.
An anomaloscope is typically used to perform color matching test [7]. Typically, for this type of assessment, observers are asked to match a yellow light with red and green lights (for a protan or deutan deficiency). The full classification of a congenital CDO is possible with a Rayleigh equation anomaloscope. It distinguishes anomalous trichromats from normal observers, dichromats from anomalous trichromats, and identifies extreme anomalous trichromats. This method is thought to give the highest accuracy compared with the above methods. Note that for assessing tritan CVD, the Moreland equation, instead of Rayleigh equation, should be recommended to match a cyan using an indigo and green [9].
Genetic testing is another method for assessing CDOs [10]. Firstly, the genetic material, i.e., DNA, should be extracted from a CDO. Subsequently, the arrangements of DNA bases known as exons are examined. In humans, the L and M photopigment characteristics are defined by six exon sequences, the first and last of which are invariant. The variability in the spectral responses of normal and abnormal photopigments is attributed to the sequence of the other four intermediary exons [11]. It should also be noted that this type of examination is usually costly and necessitates the use of professional equipment.
Following the determination of the type and severity of color deficiency, a proper image enhancement method should be used to promote the color perceptions of CDOs. Various approaches for improving color discrimination have been proposed. They are typically classified as recoloring methods, edge enhancement methods, and pattern superposition methods [3].
The original color is changed for recoloring methods to allow CDOs to better recognize it; however, such a modification usually results in a significant change of the original color. Sakamoto [12] presented a color enhancement method that employs a color palette specifically designed for protan and deutan deficiencies. These colors are known as safe colors because CDOs can easily distinguish between them. These colors are then used to replace colors in the original images. This is undeniably applicable when there aren't too many colors in the image; however, such a replacement will undoubtedly degrade the overall image quality, especially when the original image has rich colors.
For edge enhancement methods, the strategy shifts from promoting color perceptions to creating a clear and distinct difference between two color regions. A typical application of this method is shown in [13], where the borders between confusion colors were emphasized thorough increasing their luminance difference. Both CDOs and color normal observers (CNOs) can benefit from equal visual enhancement by using this method. However, the applicability of this method is somewhat limited, and it is typically not applicable to photographic images.
The goal of pattern superposition methods is to add additional information to the original image, which can be considered as a redundant color code to include a unique symbol that colorblind users can rely on when they have trouble distinguishing the colors. Patterns of various shapes will be added to the original image colors, and each pattern will have a unique meaning. [14] is a typical example of such a method. This method is clearly appropriate for images on office papers and scientific illustrations, but it is difficult to extend to photographic images.
As previously stated, both the edge enhancement and pattern superposition methods are only applicable under certain conditions. Methods of recoloring, on the other hand, can have a border range of applications. As a result, most researches endeavored in this field and resulted in a plethora of methods [15–17]. In addition, the authors proposed an advanced image enhancement method [18] for CDOs, which yielded promising experimental results.
Although all image enhancement methods claim to have overall good performance, one major drawback remains unresolved: the connection between assessment and enhancement is ignored. The majority of current image enhancement methods are designed specifically for dichromats. Only the type of deficiency is taken into account in those methods. The degree of severity, on the other hand, is frequently considered qualitatively or even ignored. This frequently exaggerates or underestimates the severity and does not make full use of the perceivable color gamut of a CDO. To address this issue, a method to link the color deficiency test result with the image enhancement method is proposed. Initially, a conversion relationship was established between a physical parameter, namely the wavelength shift (measured in nanometers) of the cone fundamental, and the color deficiency test result. Following that, the severity of a CDO was assessed and transformed into wavelength shift via the newly built conversion relationship. The wavelength shift can then be used to estimate the perceivable color gamut of a CDO. Finally, a gamut mapping algorithm was applied to map colors from the gamut of a CNO to the gamut of a CDO.
2. Color deficiency evaluation
As mentioned above, it is essential to evaluate the deficiency severity of a CDO before applying an image enhancement algorithm. There are plenty of evaluation tools available in the literature, such as FM100 test, D-15 test [19] and anomaloscope test. Since this study is specifically designed for display applications and the test results should be expressed in a quantitative score, a new testing technique, known as ZJU50Hue, was finally developed.
ZJU50Hue is a new hue-arrangement test that is based on the FM100 test. The FM100 test, as discussed in the introduction section, consists of 85 colored caps in four panels that are expected to be uniformly located in the hue direction to cover all of the hue angles. However, it is difficult to achieve an even distribution. As shown in Fig. 1, a set of FM100 samples was measured using a Konica Minolta CM-700D spectrophotometer and their color coordinates were presented in CIELAB color space [20] and CAM16-UCS color space [21]. With CIE XYZ color space (or CIE xy chromaticity diagram with equal luminance), the equal chromatic differences computed by Euclidean distance in different regions are not perceptually equal [22]. Due to the poor uniformity of the space, color spaces have been developed to improve visual uniformity. A uniform color space (UCS) usually has three dimensions, i.e., lightness, the red-green components (referred to a), and the yellow-blue components (referred to b). CIELAB is the most widely used UCS and the CAM16-UCS is thought to be the most uniform color space up to date. The circle formed by the colors is clearly more symmetrical in the CAM16-UCS than in CIELAB, indicating the superiority of CAM16-UCS. As is shown in the fourth quadrant, colors inside the area bordered by purple dots have lower saturation (lower chroma values). Moreover, the uneven distribution along the hue direction is also noticeable.
To address the aforementioned issue, a new set of color patches was created in CAM16-UCS and is shown in Fig. 2 (a). As is shown, the new set of color patches has a constant chroma value (20) and a constant lightness value (60), and the hue spacing is fixed at 4.5°. They are expected to have a more even distribution than the original FM100 samples, which are highlighted in black in Fig. 2 (a). Since all of these colors are simulated in a display, their color coordinates can be precisely controlled. The display color characterization performance will be reported in the experimental section.
Fig. 2. (a) a new set of color patches (colored dot) adapted from the FM100 caps (black dotted) and (b) remaining colors from the protanomalous and deuteranomalous observers.
According to available data [23,24], tritanopia affects only about 0.003% of the Caucasian population. When compared to protanomalous and deuteranomalous observers, tritanomaly observers have different confusion colors. It is obvious that the hue-arrangement test will be more efficient if distinguishable colors can be removed ahead of time based on the type of deficiency. As a result, colors along the tritanomaly confusion line are removed, leaving only easily confused colors for protanopes and deuteranopes. They are depicted in Fig. 2 (b).
The hue-arrangement test results were expressed using Confusion-Index (C-Index) [25], which can be used to determine the severity of a CDO. The color differences between adjacent caps arranged by an observer are defined as relative color difference vectors. The moment of inertia for relative color difference vectors is employed to calculate the major and minor radius and their angles. The C-index is derived by dividing a subject’s major radius by the major radius obtained for a perfect arrangement. A higher C-Index means a more severe color deficiency and the C-Index for a CNO is unity. The angle of the major radius can be used to estimate the type of color deficiency. It should be noted that the absolute value of the C-index can be changed by changing the test color patches for the same CDO. As a result, comparing two C- indices from different tests is not advised.
3. Conversion between the C-index and wavelength shift
To incorporate the color-vision test result into the image enhancement method, the wavelength shift of the cone fundamentals is adopted as an agency to correlate these two stages. All the assessment methods, either qualitative or quantitative, can result in a judgement of the type of deficiency as well as a rough estimation of the severity level, i.e., mild, moderate or severe. However, such a result cannot be easily utilized since there is no objective description of the severity levels. For example, even if we know a CDO has a ‘severe’ color deficiency, it is impossible to predict how much color will change. As a result, it would be appreciated if a physical parameter could be used to objectively ‘describe’ the severity of color deficiency. Hence, the wavelength shift of the cone fundamental is naturally a good candidate.
However, it is a challenge to determine the cone functions directly. Such a determination usually necessitates a large number of color matches and professional equipment [26]. To address this issue, this study proposes a new method that uses CNOs as an evaluation tool to quantify the severity of color deficiency for CDOs. This new method is done by adopting a color deficiency simulation [27] model and this model is briefly introduced in the Appendix 1 to help readers better understand this method.
Based on the simulation model, three parameters, namely the type of color deficiency, the wavelength shift of the corresponding cone spectral fundamental, and the spectral power distributions of the display primaries, are influential in the color perceptions of a CDO. Lights emitted from the display are absorbed by photoreceptors, namely the L, M, and S cones, which are distinguished by their differing peak wavelengths. The L, M, and S cone functions are shifted depending on the type and severity of color deficiency, resulting in cone responses that differ from CNOs’. This is the reason why CDOs have a different color perception from CNOs. A longer wavelength shift will lead to a more severe color deficiency, and the simulation model is completed once all the three parameters have been identified.
The color deficiency simulation model will make it easier to quantify the severity of color deficiencies. To begin, test patterns will be generated using the simulation model under various wavelength shifts and different types of deficiencies. Afterwards, CNOs were asked to perform the color-vision test using these simulated color patches, and the test results were reported using C-Index. Hence, a relationship between the C-Index and the wavelength shift was finally established. The key assumption behind this method is that the CNO's judgments on simulated images correspond to the CDO's judgments on original images. As a result, the CNOs were used as an evaluation tool to build the relationship between the test result (C-Index) and the wavelength shift. All these steps are summarized in Fig. 3.
4. Image enhancement via gamut mapping
CDOs were asked to perform the original ZJU50Hue test after the conversion relationship was established. The corresponding wavelength shift can be easily obtained by referring to the newly constructed conversion relationship.
The wavelength shift determines the color perception of a CDO and it can be utilized by the color deficiency simulation model to transform colors from the perspective of a CNO to the perspective of a CDO, i.e., the perceivable gamut of a CNO can be transformed into the perceivable gamut of a CDO. This makes it simple to comprehend the color perceptions of a CDO with a specific type and severity.
In our previous study [18], a color image enhancement method was proposed for those having color-vision deficiencies. Here, we will demonstrate how it can be combined with the color vision test result to form a personalized image enhancement algorithm.
Figure 4 depicts the workflow for determining the perceivable gamut of a CDO. Initially, the CDO was asked to perform the color vision test and the test result was reported using C-Index, which can be further transformed into the wavelength shift by referring to the conversion relationship built in Section 3. The color deficiency simulation model for this specific CDO was then determined by adopting the wavelength shift. Subsequently, the input display gamut was densely sampled in the RGB space, which in this study was a 17*17*17 RGB cube (from the view of CNO), and it was fed into the simulation model along with the wavelength shift, resulting in a new set of RGB cube (from the view of CDO). This means that, the color perceptions of a CDO can be well perceived by a CNO via the simulation model. Afterwards, the display colorimetric characterization model was applied to transform the RGB data into XYZ values, followed by the color appearance model (CAM) to obtain color appearance attributes. For simplicity, the CIELAB color space was adopted in this study. Finally, the Segment Maximum Gamut Boundary Descriptor (SMGBD) algorithm [28] was used to generate gamut boundary descriptors.
Once the CDO's gamut has been determined, a gamut mapping between the CDO's gamut and the CNO's gamut can be performed. The latter is equivalent to the display gamut. Figure 5 shows a comparison of the CNO's gamut and the CDO's gamut. It is clear that as the wavelength shift increases, the gamut of CDO shrinks, resulting in limited color perceptions. At this point, the image enhancement problem can be considered as a gamut mapping problem.
Fig. 5. The gamuts comparison between CNO and CDOs. Two CDOs of protanomalous observers having 10 nm and 15 nm wavelength shifts are included.
There are plenty of available gamut mapping algorithms available in the literature [29,30]. In this study, a simplified version of the well-known SGCK (Chroma-Dependent Sigmoidal Lightness Mapping Followed by Knee Scaling Toward the CUSP) algorithm [28] was adopted and is illustrated in Fig. 6. A nonlinear knee function was applied to all source colors. If the source color was within 90% region of the destination gamut (called the core region), it would be preserved otherwise mapped towards the focal point E, which has the same lightness of the original color. In other words, the line segment EPo was mapped to the line segment EPd, while colors in the core region remained unchanged. The function adopted to perform such mapping is given in Eq. (1).
Fig. 6. Mapping towards the lightness axis. Po is the original color in the source gamut boundary, and Pd is the mapped color in the destination gamut boundary. E is the mapping centre on the lightness axis that has the same lightness value as point Po. The length of EPs equals to 90% the length of EPd.
Fig. 7. The comparison between the original image and its enhanced one. The wavelength shift is set at 10 nm of a protanomalous observer
After gamut mapping, all colors were within the destination gamut that a CDO can well perceive. This output image, however, is from the perspecve of a CDO, and it had to be converted back to make a copy from the perspective of a CNO. This is easily accomplished by inverting the simulation model, as shown in Appendix 1. Fig 7 depicts an example of the enhanced image as well as the original image from the perspectives of both the CDO and the CNO.
5. Experiments and results
Physical experiments were conducted in this study to validate the effectiveness of this method. The first one is to establish the conversion relationship between the wavelength shift and the color-vision test result (C-Index). The second one is to test the performance of this proposed method using CDOs.
Twenty participants including ten CNOs, two protans and eight deutans were recruited from Zhejiang University using an online social networking platform. The selected applicants have an average age of 25 ± 4.23 (std). The CNOs are consisting of 4 males and 6 males, and all of them passed the Ishihara test to ensure they are not color-weak. The CDOs consist of 8 males and 2 females. All procedures were approved by the Zhejiang University Human Research Ethics Committee. Participants gave written informed consent and were reimbursed for their time.
5.1 Establishment of the conversion relation
A pilot study was carried out in accordance with Section 2 to correlate the C-Index with the wavelength shift. The original ZJU50Hue pattern was sent into the simulation model with various wavelength shifts ranging from zero to maximum in this study. With those simulated images, CNOs were asked to perform the ZJU50Hue test. Their results were then analyzed using the C-Index, and a relationship between the C-Index and the wavelength shift was finally established.
Ten CNOs participated in the pilot study. Each of them was asked to judge the test patterns of various wavelength shifts including 5, 10, and 15 nm for two common types of color deficiency, namely protan and deutan. As a result, a total of six tests (= three wavelength shifts ×two types) were included. The tests were carried out in a darkened room with all lights turned off. In this study, a NEC PA301W display was used, and it was well characterized using a GoG model [31], with an average color difference less than a CIELAB unit when tested with the Macbeth Color Checker. The display white was set at D65 @ 100 cd/m2, and the background of the testing interface was fixed at L*a*b* = [50,0,0] to ensure a consistent white adaptation. The whole interface is illustrated in Fig. 8.
Fig. 8. The experimental setup using ZJU50Hue test. Observers were asked to pick color patches from the row below and sort them in the upper row. Only one panel of color patches (red to green) is illustrated.
The subjects were asked to sit in front of the display at an 80 cm distance during the experiment. Each disc's angular subtense was set at 2°. One thing to keep in mind is that in this experiment, there was no fixed cap. In this scenario, subjects were instructed to choose any disc from the lower row and place it in the upper row at random. Subsequently, they chose the disc with the closest color match and placed it next to it. All tasks were carried out using a mouse. In addition, subjects can ‘insert’ the disc into two adjacent discs during each placement. This offers more freedom to the subjects compared with the original FM100 test, which uses a fixed starting point. Each subject did only one session at a time for a given wavelength shift and deficiency type. This is to avoid eye fatigue for excess assessments. In total, each subject completed 6 sessions (3 wavelength shifts × 2 types of deficiency) in a random order, which lasted about 2 hours including a 30 mins break.
The test results made by CNOs were utilized to model the relationship between the C-Index to wavelength shift. CNOs were used as simulated CDOs in this study, and they were requested to perform the ZJU50 Hue test under the condition of three wavelength shifts (5, 10, and 15 nm). Different wavelength shifts represent the extent of severity, and each corresponds to a particular C-Index, which represents the mean performance of the simulated CDOs. Therefore, CDOs under each wavelength shift were evaluated using C-Index and the mean value represents the typical performance of a CDO having that specific deficiency severity.
Initially, two-way analysis of variance (ANOVA) and post-hoc tests were conducted to reveal the statistical differences between wavelength shift and type of color deficiency. Table 1 gives the ANOVA results. It showed that wavelength shift had a significant effect on the deficiency severity (C-Index), implying the color discrimination capability is different for simulated CDOs having different deficiency severities. In contrast, testing results were not significantly different for different types of deficiency (protan or deutan). However, it can be still observed from Fig. 9 that the simulated deuteranope tends to have a larger C-Index compared with the simulated protanope. This might be due to learned luminance cues that, reds are substantially dimmer than greens to protanopes [32].
Fig. 9. The testing results for CNOs on ZJU50 hue test at 3 specific wavelength shifts (5, 10 and 15 nm). The experiment was aimed to simulate protanopes and deuteranopes under three severity levels (mild, moderate and severe). The red and green lines represent results for simulated protanopes and deuteranope, respectively. The error bar represents a standard deviation.
Table 1. Results of ANOVA for wavelength shift and type of color deficiency
Fig. 9 depicts the testing results. A red and a green curve, representing simulated protanope and deuteranope results, were included. To reflect the scatter of data, an error bar representing one standard deviation has been added. As shown, the C-Index rises as the wavelength shift increases, indicating a more severe deficiency. Most of the time, the green line is on top of the red line for the same wavelength shift, indicating poor color discrimination. Furthermore, the slope grows larger as the wavelength increases, implying that the severity changes quickly for a larger wavelength shift.
One may also notice that the scatter of the results is not negligible and grows larger as the wavelength shift increases. This might have to do with nature of the hue-arrangement test. This type of experiment is based on the fact that CDOs are easily confused by adjacent colors. And as the wavelength shift increases, more colors will be confused. Any sorting is possible for those muddled colors. As a result, for a large wavelength shift, there will be more sorting permutations, resulting in a scattered testing result.
5.2 Validation using CDOs
Two psychophysical experiments were carried out to validate the effectiveness of this method. The first was the same ZJU50Hue test, but with enhanced color patches. When a CDO completed the original ZJU50Hue test, the wavelength shift was determined using the relationship curve developed in the pilot study. Following that, the enhanced color patches were created by employing the gamut mapping algorithm described in section 4. The observer was asked to repeat the task, but this time with the enhanced color patches. If the proposed method is valid, a CDO's color perception will be improved, resulting in a better testing result, i.e., a lower C-Index.
Another experiment was using images chosen from the Ishihara test book, as illustrated in Fig. 10. These test images are widely accepted as a good evaluation tool to determine the color discrimination capability of a CDO. The CDO was asked to choose a better one between the original image and its enhanced reproduction in terms of color discrimination.
Fig. 10. The Ishihara test images. All these images contain easily confused colors. These are the original images without enhancement.
Ten CDOs including two protanomalous (PA) and eight deuteranomalous (DA) observers participated in this experiment. Their degrees of severity varied from mild to severe according to the Ishihara test result. The experimental setup was the same as the pilot study.
Experimental results were summarized in Table 2. The deficient type and the wavelength shift were obtained from the ZJU50Hue test by referring to the conversion relationship between C-Index and wavelength shift. Initially, a paired t-test was implemented to the C-index before and after the implementation of color enhancement, and the result (p = 0.01 < 0.05) demonstrated the enhancement algorithm had a significant impact on color discrimination. A down arrow means the C-Index reduces when using the enhanced test colors, i.e., CDOs had a better performance on sorting the enhanced color caps than the original ones. This indicates an increase in color discrimination. The portion of enhancement is calculated as the number of selections of the enhanced images divided by that of total images. These test images are illustrated in Fig. 10. As is shown, there were 14 images adopted in this experiment. For each CDO, an enhanced version of these images will be generated. For example, ten of them gave better color discrimination than their original ones, the portion of enhancement is 71.43% $(\textrm{i}.\textrm{e}.\textrm{}\frac{{10}}{{14}} \times 100\%)$. As is shown in Table 2, most of the enhanced images were selected by the observers, indicating that they had better discrimination on the enhanced images over the original ones. This again supports the effectiveness of the proposed method.
Table 2. Test results for the psychophysical experiments. A down arrow means the C-Index drops when using the enhanced test colors, indicating an increase in color discrimination. The portion of enhancement is calculated as the number of selections of the enhanced images divided by that of total images. A Slash means this observer did not participate in this part
6. Discussion
A comprehensive study on image enhancement for CDOs was conducted, including color deficiency assessment and image enhancement. The wavelength shift was used to connect both stages, which served as an indicator to reflect the severity of a CDO. It is obvious that the color deficiency simulation is important to the overall study, and it is also recommended to test other recently proposed simulation models [33] to obtain the most reliable result.
In the evaluation section, a new hue-arrangement test adapted from the FM100 was used, and the results were analyzed using the C-Index. The color deficiency simulation was used to establish a conversion relationship using CNOs. As a result, the severity of a CDO expressed using the C-Index was converted into a physical parameter, namely the wavelength shift. The scatter of the pilot study results using CNOs represents the accuracy of this method, and a tool with a smaller dispersion is desired. The selection of assessment method and the quantitative scoring technique are the two major factors influencing the accuracy of assessment results in this section. Other tools, such as the color matching techniques can also be applied to validate the testing results.
The gamuts of CDOs were analyzed using their wavelength shifts in the image enhancement section. Colors were then mapped from the CNO's gamut to the CDO's gamut via a gamut mapping technique. Choosing an appropriate gamut mapping algorithm is critical in this section. Different gamut mapping algorithms were developed with different considerations in mind, and it is recommended that one be chosen to suit the gamut shape CDOs.
Only deficiencies with a moderate to severe deficiency participated in this experiment. As a result, it is preferable to include more CDOs with mild deficient severity. However, since CDOs of mild severity had a relatively larger perceivable color gamut, it can be inferred that the performance of this method should be better on them.
To sum up, the limitations of this study can be ascribed as follows. To begin with, this study exclusively considers inherited CVD. The genesis of acquired deficiency, on the other hand, was completely different. As a result, the technique developed in this study cannot be applied to them directly. Secondly, the newly developed color vision test was focused on protan and deutan CVDs, and it needs to be updated to include tritan CVDs. Thirdly, the study had a small number of participants. To properly investigate the performance of the proposed technique, a larger-scale experiment is preferred.
It is also worth noting that the method proposed in this study serves as a template for image enhancement for CDOs. A more advanced simulation model, a more accurate assessment method, or a more sophisticated gamut mapping algorithm can produce a better result.
7. Conclusion
A comprehensive study on image enhancement for CDOs was conducted. It consists of three stages: the evaluation of color severity and the gamut mapping for image enhancement. Two psychophysical experiments were carried out to validate this method, which show that it is a promising solution for enhancing images for CDOs.
Appendix 1
To ensure better comprehension of this study, an introduction to the simulation model is provided. As suggested by the opponent-color theory [29], the trichromatic theory adopting three photoreceptors is valid at the photoreceptor level and should be further processed by the suprathreshold transformation as given in Eq. (2),
As discussed in the introduction section, the wavelength shift of the cone fundamentals leads to color deficiency, which occurs at the retinal level. This indicates that, the neural connections that connect photoreceptors to the rest of the visual system are unaffected, which is the basic assumption of the color deficiency simulation model. As a result, the only difference between a color normal observer and a color dicient observer lies in the peak wavelength of the $L(\lambda )$, $M(\lambda )$ and $S(\lambda )$ functions, depending on the type and severity of color deficiency.
Hence, the opponent color attributes can be simply accomplished by projecting the spectral power distributions ${\varphi _R}(\lambda )$, ${\varphi _G}(\lambda )$ and ${\varphi _B}(\lambda )$ of the display RGB primaries onto the set of basis functions $WS(\lambda )$, $YB(\lambda )$ and $RG(\lambda )$ of the opponent color space, as shown in Eq. (3).
Taken the above equations in mind, the processing procedures of the visual system can be simplified using Eq. (4):
As a result, the color deficiency simulation model can be easily expressed using a three-by-three matrix, as shown in Eq. (6). As is shown, the model convers colors from the view of a CNO to the view of a CDO. Meanwhile, the opposite version of the simulation model can be obtained by simple matrix operations shown in Eq. (7). It should be noted that, the RGB values for both models are linear values, i.e., values with gamma encoded.
Funding
Fundamental Research Funds for the Provincial Universities of Zhejiang (GK219909299001-019); National Natural Science Foundation of China (61775190).
Acknowledgment
The authors would like to acknowledge support from OPPO Guangdong Mobile Communications Co. Ltd.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. T. Rajalakshmi and S. Prince, “Physiological modeling for detecting degree of perception of a color-deficient person,” J. Eng. Med. 231(4), 276–285 (2017). [CrossRef]
2. J. Birch, “Worldwide prevalence of red-green color deficiency,” J. Opt. Soc. Am. A 29(3), 313–320 (2012). [CrossRef]
3. CIE, “Enhancement of Images for Colour-Deficient Observers,” CIE 240:2020 (2020).
4. L. H. Hardy, G. Rand, and M. C. Rittler, “Tests for the Detection and Analysis of Color-Blindness. I. The Ishihara Test: An Evaluation,” J. Opt. Soc. Am. 35(4), 268–275 (1945). [CrossRef]
5. L. H. Hardy, G. Rand, and M. C. Rittler, “H-R-R Polychromatic Plates,” J. Opt. Soc. Am. 44(7), 509–523 (1954). [CrossRef]
6. D. Farnsworth, “The Farnsworth-Munsell 100-Hue and Dichotomous Tests for Color Vision,” J. Opt. Soc. Am. 33(10), 568–578 (1943). [CrossRef]
7. S. Fmsa and S. Dain, “Clinical colour vision tests,” Clin. Exp. Optometry 87(4-5), 276–293 (2004). [CrossRef]
8. B. C. Regan, J. Reffin, and J. Mollon, “Luminance noise and the rapid determination of discrimination ellipses in color deficiency,” Vision Res. 34(10), 1279–1299 (1994). [CrossRef]
9. J. D. Moreland and J. Kerr, “Optimization of a Rayleigh-type equation for the detection of tritanomaly,” Vision Res. 19(12), 1369–1375 (1979). [CrossRef]
10. J. Nathans, “The evolution and physiology of human color vision: insights from molecular genetic studies of visual pigments,” Neuron 24(2), 299–312 (1999). [CrossRef]
11. L. T. Sharpe, A. Stockman, H. J Agle, and J. Nathans, “Opsin genes, photopigments, color vision, and color blindness,” in Color Vision: From Genes to Perception (Cambridge University, 1999), pp. 3–51.
12. T. Sakamoto, “Image color reduction method for color-defective observers using a color palette composed of 20 particular colors,” Proc. SPIE 9395, 939514 (2015). [CrossRef]
13. T. Sakamoto, “Edge enhancement filter for people with protanopic and deuteranopic vision,” in Proceedings of AIC(2012), pp. 406–409.
14. P. Hung and N. Hiramatsu, “A colour conversion method which allows colourblind and normal-vision people share documents with colour content,” in Proceedings of the 27th Session of the CIE(2011), pp. 229–239.
15. S. Hira, A. Matsumoto, K. Kihara, and S. Ohtsuka, “Hue rotation (HR) and hue blending (HB): Real-time image enhancement methods for digital component video signals to support red-green color-defective observers,” J. Soc. Inf. Display 27(7), 409–426 (2019). [CrossRef]
16. J. I. You and K. Park, “Image Processing with Color Compensation Using LCD Display for Color Vision Deficiency,” J. Display Technol. 12(6), 562–566 (2016). [CrossRef]
17. G. R. Kuhn, M. M. Oliveira, and L. A. F. Fernandes, “An Efficient Naturalness-Preserving Image-Recoloring Method for Dichromats,” IEEE Trans. Visual. Comput. Graphics 14(6), 1747–1754 (2008). [CrossRef]
18. L. Xu, Q. Li, X. Liu, Q. Xu, and M. R. Luo, “Gamut mapping based image enhancement algorithm for color deficiencies,” Biomed. Opt. Express 12(11), 6882–6896 (2021). [CrossRef]
19. B. Foutch, J. Stringham, and V. Lakshminarayanan, “A new quantitative technique for grading Farnsworth D-15 color panel tests,” J. Mod. Opt. 58(19-20), 1755–1763 (2011). [CrossRef]
20. CIE, “Colorimetry. 2nd Ed.,” in CIE Publ. No. 15.2 (Vienna: Central Bureau of the CIE, 1986).
21. C. Li, Z. Li, Z. Wang, Y. Xu, M. R. Luo, G. Cui, M. Melgosa, M. H. Brill, and M. Pointer, “Comprehensive color solutions: CAM16, CAT16, and CAM16-UCS,” Color Res. Appl. 42(6), 703–718 (2017). [CrossRef]
22. M. Rodriguez-Carmona, “Genetics of Photoreceptors, Genetics and Color Vision Deficiencies, Genes of Cone Photopigments,” in Genes and Cones (Springer Berlin Heidelberg, 2015).
23. W. D. Wright, “The sensitivity of the eye to small colour differences,” Proceedings of the Physical Society (1926-1948) 53, 93 (1941).
24. C. Rigden, “The eye of the beholder-designing for colour blind,” British Telecommunications Engineering 17, 291–295 (1999).
25. A. J. Vingrys and P. E. King-Smith, “A quantitative scoring technique for panel tests of color vision,” Invest. Ophthalmol. Visual Sci. 29, 50–63 (1988).
26. W. S. Stiles and J. M. Burch, “N.P.L. Colour-matching Investigation: Final Report (1958),” Opt. Acta 6(1), 1–26 (1959). [CrossRef]
27. G. Machado, M. Oliveira, and L. Fernandes, “A Physiologically-Based Model for Simulation of Color Vision Deficiency,” IEEE Trans. Visual. Comput. Graphics 15(6), 1291–1298 (2009). [CrossRef]
28. CIE, “Guidelines for the evaluation of gamut mapping algorithms,” CIE Pub.156 (2003).
29. L. Xu, B. Zhao, and M. R. Luo, “Colour gamut mapping between small and large colour gamuts: Part I. gamut compression,” Opt. Express 26(9), 11481–11495 (2018). [CrossRef]
30. L. Xu, B. Zhao, and M. R. Luo, “Color gamut mapping between small and large color gamuts: part II. gamut extension,” Opt. Express 26(13), 17335–17349 (2018). [CrossRef]
31. R. S. Berns, “Methods for characterizing CRT displays,” Displays 16(4), 173–182 (1996). [CrossRef]
32. H. Brettel, F. Viénot, and J. Mollon, “Computerized simulation of color appearance for dichromats,” J. Opt. Soc. Am. A 14, 2647–2655 (1997). [CrossRef]
33. H. Yaguchi, J. Luo, M. Kato, and Y. Mizokami, “Computerized simulation of color appearance for anomalous trichromats using the multispectral image,” J. Opt. Soc. Am. A 35(4), B278–286 (2018). [CrossRef]
34. C. Ingling and B. Huong-Peng-Tsou, “Orthogonal Combination of the three visual channels,” Vision Res. 17(9), 1075–1082 (1977). [CrossRef]
