1. Introduction

Modelling human color perception is the main mission of color science. Multiple color appearance models (CAMs) which generate perceptual attributes of colored stimuli have been developed, such as the Nayatani et al. model [1], the Hunt model [2], CIECAM97s [3], CIECAM02 [4] and CAM16 [5], some of which are recommended by the Commission Internationale de l’Éclairage (CIE). In these models, from the optical characterization of the stimulus and background in terms of spectral radiance or tristimulus values, a number of processing steps mimicking the human visual system are defined to output a set of absolute (brightness, colorfulness, saturation and hue) and relative (lightness, chroma) visual correlates of the stimulus. Though being widely applied in the fields of printing and media reproduction, the majority of existing CAMs still have certain limitations in their applications as they assume a uniform stimulus (typically with a defined angular extent between 2 and 10°) seen on a uniform background (typically extending for 10° from the edge of the stimulus) and categorized surround [6], while real life stimuli, such as street lighting or outdoor billboards, are often observed in much more complex environments.

To extend the applications of traditional CAMs to more complex scenarios, some image color appearance models have also been created. These models are capable of predicting different spatial color perception effects such as simultaneous contrast, crispening or spreading, and are applied for image quality assessment [7], high dynamic range (HDR) image rendering [8–11] and image enhancement [12,13]. While the iCAM models [7,8] and the Reinhard et al. model [9] are mainly based on fundamental CAMs such as CIECAM02, Meylan et.al [10] and Benoit et al. [11] proposed models which are more physiologically based, with tone-mapping operators inspired by retinal processing mechanisms. Though the filter kernel size influences the performance of such models [14], the physiological background behind the choice of the kernel size has not been clearly justified [13]. Moreover, Kolas et al. [12] developed a framework for spatial color algorithms based on the Retinex theory by Land [15], and Provenzi et al. [13] suggested a color correction algorithm with a series of mathematical assumptions based on the human visual system. However, these models were not developed explicitly for predicting color appearance and the visual attributes, such as brightness, are not explicitly extracted from the models.

Applying the previously mentioned non-imaging and imaging color appearance models to self-luminous stimuli, such as light sources present in general lighting scenes, is, however, not straightforward. The absolute spectral radiance of the stimulus can be independent from that of the background leading to an ambiguity in the definition of and normalization to the white point. It has also been shown that these models underestimate the Helmholtz-Kohlrausch (H-K) effect [16]. To overcome those limitations, some CAMs have been developed for self-luminous stimuli [16,17]. One of them, CAM18sl [16], has been implemented in a wide range of applications such as visual gloss, the CIE UGR for discomfort glare, the CIE gray-scale calculation for self-luminous devices, the requirements for traffic signalization [18], as well as to predict the age effect in brightness perception for saturated (red and blue) stimuli [19]. Nevertheless, CAM18sl has the same limitations as the classical CAMs mentioned before, being only applicable to situations characterized by a uniform stimulus and background. As these limitations can be resolved by using image-based color appearance models, the application of the iCAM model to self-luminous scenes has been investigated [14]. Unfortunately, it has been shown that the model has some drawbacks, such as an underestimation of H-K effect and the stimulus size effect and an overestimation of the brightness perception of self-luminous stimuli surrounded by a dark background [14]. All these observations call for a more comprehensive color appearance model applicable for complex scenes including light sources, which might be called a lighting appearance model (LAM).

A first step towards a LAM is the development of an image-based model for brightness perception of self-luminous stimuli in complex situations. Having an important role in lighting and display applications, more particularly in defining glare level or contrast threshold [18,20–22], multiple studies have been performed to investigate the impact of the complex viewing environment on the brightness perception of a stimulus. It has been shown that different properties of the background, such as its size and the spatial compositions, have a significant impact [21,23–26]. Stevens [23] showed that the area of the inhibiting field determined the degree of brightness decrease of the object. Later, Sun et al. [24] also confirmed that by increasing the size of the luminous background, the brightness of the stimulus would decrease. The separation between the target and background stimulus which are part of the background also appears to influence how the brightness of the target stimulus is perceived: the smaller the separation, the higher the impact on the stimulus brightness [25]. Whittle [21] also found out that the introduction of a thin outline or a hue change between the stimulus and background could reduce the crispening effect. Carter et al. [26] pointed out that the brightness of a stimulus could also be impacted by the luminance changes of an extended area in the background.

In addition, Reid and Shapley proposed a model which considers the surrounding environment’s contribution through modelling brightness contrast and assimilation effects [27]. In these effects, the stimulus brightness changes in the opposite and in the same direction as the background brightness, respectively. Shevell et al. introduced a two-stage model which simulates the neural mechanisms of brightness induction using the retinal stimulation from the target stimulus area and its adjacent area, in conjunction with the neural response from the remote area in the field of view [28]. Kingdom and Moulden also developed a multi-channel model to simulate different brightness phenomena using multiple spatial scale filtering [29], which has then been extended with the two-dimensional brightness model by McArthur et al. [30]. These models are capable of predicting various well-known brightness phenomena, such as simultaneous contrast [27], in complex situations. However, the models either only investigated the phenomena for a small field of view of only a few degrees [27,28] or the models are not verified with human perception experiments [29,30]. McCann [31] and Rudd [32] also proposed strongly physiological-based lightness computation models, which work well for various spatial conditions for neutral scenes, yet, applying these models to self-luminous stimuli might face the same issues as traditional CAMs due to the ambiguity of defining the reference white point for self-luminous scenes.

In order to extend a CAM for object colors to a LAM including light sources, mastering the perception of the brightness of light sources in a complex scene is a first crucial step. This study addresses the question how different parts of the background influence the brightness perception of a neutral self-luminous stimulus. An image-based brightness model is presented which combines the merits of classical appearance models applicable to self-luminous sources with image-based approaches in which complex backgrounds are considered. As a first step, a symmetrical ring-shaped luminous background element characterized by a well-defined gap towards the stimulus is considered. Psychophysical brightness matching experiments are conducted to investigate how the distance (up to 16° from the edge of the stimulus), the area and the luminance of a neutral luminous ring-shaped background influences the brightness perception of a neutral stimulus. The experimental data are modelled using a pixel-by-pixel image-based approach including cone-fundamental weighted spectra radiance, stray light, cone compression, a receptive field post-receptor organization and an adaptive shift. A connection to the CAM18sl model is made by extending the width of the luminous background until a uniform background is reached. Although the experimental setting is still relatively simple (neutral circular stimuli, neutral and ring-shaped induction areas), the experiments allow to develop and test a next-generation image-based brightness model for self-luminous stimuli.

2. Experiment

2.1 Experimental setup

The stimuli used in the experiments were created with PsychToolbox [33–35] in MATLAB [36] and displayed on an EIZO ColorEdge PROMINENCE CG3145 monitor. The reference stimulus has a field of view (FOV) of approximately 10°, while the display subtends a FOV of 82°${\times} $ 49° with the observer being seated at a distance of 40 cm to the screen. A black shield is placed in the middle of the screen to ensure that the reference and the test stimuli are visually separated and do not influence the perception of one another. The set-up of the experiment is illustrated as in Fig. 1.

 

Fig. 1. Pictures of the experiment set-up. (a) A view of the experiment set-up showing the shield and the screen; the reference stimulus is displayed at the left half and the test stimulus at the right half. (b) The top view of the setup. The blue circles indicate where the observer should position their head when viewing the experimental scenes.

Download Full Size | PDF

The reference stimulus consists of a neutral gray circle with a 10° diameter and reference luminance L_10,ref of 65 cd/m². The CIE 2006 10°chromaticity coordinates of (x₁₀ = 0.31, y₁₀ = 0.32), which is similar to the chromaticity of CIE illuminant D65.

The test scene was a central test stimulus having the same size as the reference stimulus surrounded by an additional luminous area. To ensure an equal impact of the distance and the luminance of the additional element to the central stimulus from all directions, the luminous area was chosen as a uniform ring. The test stimulus and ring had the same CIE 2006 10° chromaticity as the reference stimulus, but different luminance levels. The test stimulus was initially shown at either a low starting luminance of 35 cd/m² or a high starting luminance of 335 cd/m², so that at the beginning, the test stimulus would appear clearly different in perceived brightness compared to the reference stimulus. Three ring luminance levels were used in this study: L_10,ring = 90 cd/m², L_10,ring = 335 cd/m² and L_10,ring = 1200 cd/m², which covered the low, medium and high luminance ranges of the screen. Further, the ring was presented with three possible thicknesses of 0.33 cm, 0.67 cm and 1.00 cm (average angular widths of 0.25°, 0.49° and 0.73°, respectively) and at 4 different angular gaps with respect to the outer edge of the stimulus: 1.2°, 6.4°, 11.3° and 16.1°. Each ring thickness was used in combination with all three ring luminance levels at all four angular gaps.

To avoid positional bias and luminance starting point bias, the test stimuli were shown both to the left and to the right of the reference stimulus and were initially displayed at two different starting luminance levels. To account for the matching errors made by each observer, each experimental sequence included two test stimuli without any ring around it and shown at two starting luminance levels. Two repeated scenes were also included for checking the intra-observer variability. The test sequence for each observer at each experimental session was randomized to avoid ordinal bias.

In each experimental phase, first, each observer was asked to perform a trial session that included 5 random stimuli to get used to the procedure, and then one official session for each ring thickness at each ring luminance level. In each official experimental session, 22 test scenes were shown (4 distances of the ring ${\times}$ 2 starting points ${\times}$ 2 reference positions + 2 scenes to test matching error + 4 repeated scenes for intra-observer variability).

2.2 Experimental procedure

Visual data was collected using a brightness matching method. Starting with 5 minutes of adaptation, the observer was seated in a dark room and looking at a scene randomly chosen from the set of test scenes while receiving instructions.

The observer was asked to change their gaze position such that they would always maintain a fixed distance of 40 cm to the screen and look perpendicularly at the center of the reference and the test stimulus with binocular view, one at a time. The task given to the observer was to adjust the brightness of the test stimulus such that it could be visually perceived as equally bright as the reference stimulus. To perform the brightness matching task, the observer informed the experimenter whether they preferred to adjust the stimulus intensity with a keyboard themselves or they would rather instruct the experimenter orally. The screen is controlled with the 10-bit signal and the observer had the option to change the stimulus brightness with a coarse adjustment (5 RGB levels) and by fine-tuning (1 RGB level). These increment steps allow the observers to perform the adjustments with consistent perceptual brightness steps using the coarse adjustments while still having an option to fine-tune the match. The average time for each observer to finish one matching session was between 20 and 45 minutes.

The observer panel included 20 subjects (13 males and 7 females) aged between 21 and 61 years old with an average of 30.5 years. All observers had normal or corrected to normal color vision as tested by the Ishihara 24-plate test.

Once the matching result was obtained for each observer, the spectral radiance of the matched stimulus was measured with a JETI Specbos 1211 spectroradiometer. The measured area covered the central 25% of the stimulus area. Good uniformity of less than 2% difference between the minimum and the maximum luminance values was found; the screen did not show pixel cross-talk. To obtain the spectral radiance of the thin ring, the measurement was performed on an arbitrary larger area on the display having the same RGB values as the ring, as the thickness of the ring was smaller than the measurement spot of the spectroradiometer.

The short, medium and long cone-weighted and scaled spectral radiance values ${L_\rho },{L_\gamma },{L_\beta }$ of the matched stimulus are computed from the measured spectral radiance ${L_{e,\lambda }}(\lambda )$ and the set of cone fundamentals $\overline {{l_{10}}} ,\overline {{m_{10}}} ,\overline {{s_{10}}}$ for 10° stimuli as provided by the CIE in 2006 [37–39]. The normalization coefficients were chosen such that for a D65 stimulus, the cone-weighted and scaled spectral radiance values are identical and equal to the CIE 2006 10° luminance value:

The cone-weighted spectral radiance values of the reference stimulus and the ring are calculated similarly.

As the stimuli and the rings have a chromaticity close to that of the D65 illuminant, their three cone-weighted values are almost identical with a difference of less than 2% between the minimum and the maximum values. Therefore, the arithmetic mean, denoted as ${L_\alpha }$, is used to describe the test and the reference stimulus ${L_{\alpha ,test}}$ and ${L_{\alpha ,ref}}$, and of the ring ${L_{\alpha ,ring}}$, respectively. As long as neutral stimuli are targeted, this approach allows a reduction of the number of cone-weighted input values for the model [40]. As the cone-weighted spectral radiance value is a measure for the absorption rate of photons in the cones of the retina, these values can be considered as cone excitations.

3. Results

3.1 Observer variability

The average intra- and inter-observer variability were calculated by taking the arithmetic mean over the observers of the standardized residual sum of squares (STRESS) obtained for each observer. The value can be used to analyze the agreement between two sets of data, where two sets with the perfect agreement would result in a STRESS value of zero [41]. The inter-observer variability was calculated as the average of STRESS between the data collected by each individual observer to the average observer, and the intra-observer variability was calculated as the average of STRESS between the matches of the repeated scenes from each individual observer. The STRESS values were also calculated to check the agreement between the data collected from different reference positions, starting luminance levels and the matching error variability.

The average intra- and inter-observer variability are calculated to be 18% and 21%, respectively, which are similar ( [24,40,42,43]) or better ( [44,45]) than the results from previous literature regarding brightness experiments using matching or magnitude estimation methods. The matching error average of roughly 12% indicates a baseline of the matching reliability.

To check if the choice of the reference position and the starting luminance level have an influence on the brightness matching results, a Kruskal-Wallis test was done, which shows that there are statistical differences between the dataset collected from different reference positions (H(1) = 13.14, p-value = .0007) and from different starting luminance levels (H(1) = 34.41, p-value = 4.47e-09).

Finally, an average matching result for each test scene done by each individual observer is obtained by taking the arithmetic mean of ${L_\alpha }$, of the matched stimuli from different test scene positions and different starting luminance levels. Then, the arithmetic mean of the average results from all individual observers is computed for each test scene to present the final average matching result.

3.2 Brightness matching results

The ratio of the average cone excitation of the test stimulus ${L_{\alpha ,test}}$ over the cone excitation of the reference stimulus ${L_{\alpha ,ref}}$ (surrounded by a dark background) as a function of the gap or distance from the ring to the stimulus is illustrated in Fig. 2.

 

Fig. 2. The ratio of the average cone weighted spectral radiance of the matched stimuli as a function of angular distance from the stimulus edge to the ring with different ring thicknesses and ring luminance levels: (a) Without error bars. (b) With error bars. The error bars represent the standard error.

Download Full Size | PDF

As ${L_{\alpha ,test}}$ is always higher than ${L_{\alpha ,ref}}$, the results show that the observers always select a higher radiance of the test stimulus to obtain a brightness match with the reference stimulus. This implies that by introducing a luminous ring in the visual field, whatever the luminance, gap or thickness, the perceived brightness of the center stimulus will darken. The influence of the ring luminance is occurrent as higher ring luminance levels result in a higher matching value, indicating a higher inhibiting effect. In addition, the general tendency is that when the ring is at a closer distance, the impact of the ring on the stimulus brightness is higher than when the ring is at a further distance. Even at the furthest distance of 16.1°, the ratio ${L_{\alpha ,test}}/{L_{\alpha ,ref}}$ is still greater than 1. However, the impact of distance seems to disappear when the ring luminance level is low, as the matching results remain almost the same for all distances.

Finally, the thickness of the ring is shown to have a certain impact on the perceived brightness of the central stimulus. The effect is highest at the closest ring distance. It becomes obvious that the rings with a thickness of 0.67 cm and 1.00 cm have quite a similar effect, while the impact of the ring with the smallest thickness (0.33 cm) is much less. This was confirmed by a Kruskal-Wallis test to check the statistical difference between the dataset obtained with the ring thickness of 0.33 cm and the ring thickness of 1.00 cm (H(1) = 3.85, p-value = .049), and between the dataset obtained with the ring thickness of 0.67 cm and the ring thickness of 1.00 cm (H(1) = 0.05, p-value = .82).

4. Modelling

Based on the experimental results, an image-based brightness model inspired by the physiology of the retina is proposed to predict the impact of distance, the area and the luminance of the ring on the brightness perception of the stimulus. Physiological based retinal models [46,47] adopt the following workflow: the cone excitation is compressed and the output is transmitted directly to a bipolar cell, representing the center signal. The horizontal cells connect several adjacent photoreceptors within the receptive field of the bipolar cells, creating a surround signal. The center signal and surround signal are subtracted and transmitted. Similar processes occur in the retina’s inner plexiform layer leading to a response of the ganglion cells which is sent to the brain via the optic nerve [6]. This general physiological based workflow is the main inspiration for the image-based brightness model that is described in the following section.

4.1 Proposed model framework

In this section, an image-based and retinal inspired model is proposed according to the framework illustrated in Fig. 3. The model starts from the pixel by pixel ${L_\alpha }$ map. In the first processing step, scattering in the eye is modelled, resulting in a slightly blurred image. This step is followed by a compression, the calculation of a receptive field based feedback signal and an adaptive shift. Finally, a brightness related correlate is calculated.

 

Fig. 3. The image-based brightness prediction framework illustrated by grayscale images (with the values in each image scaled between the minimum and the maximum pixel values of the image). The stimulus size and the ring size are adjusted for illustration purpose. The * symbol represents the convolution of the image with a filter kernel.

Download Full Size | PDF

For the development of the model, a set of images has been created corresponding to each experimental scene. A resolution of 1280 ${\times}$ 675 pixels, corresponding to an angular resolution of around 4’, has been chosen. The local ${L_\alpha }$ value (${L_{\alpha ,test}}$, ${L_{\alpha ,ref}}$, ${L_{\alpha ,ring}}$ and ${L_{\alpha ,dark}} = 0$) is attributed to each corresponding pixel. The central stimulus used in the experiments corresponds to a diameter of 135 pixels in the image. This image is called the average cone excitations map ${I_\alpha }(x,y)$ (Fig. 4).

 

Fig. 4. An example of the input image: (a) The full image. (b) The value of each pixel from a line cut through the image.

Download Full Size | PDF

4.2 Impact of scattering in the eye

The presence of a luminous ring in the vicinity of a circular stimulus can also generate a veiling luminance at the stimulus, which might influence the matching results and which needs to be corrected for. In fact, in a human eye, a point source is not imaged on the retina as a single point but is spread out. This is caused by several optical effects of the eye [48]. The most common way to describe the distribution of these scattering effects is by introducing the point spread function (PSF) as defined by CIE [49]:

Note that PSF is expressed in units $s{r^{ - 1}}$ and that the values are normalized. Indeed, the illuminance at the cornea created by the original point source should be equal to the illuminance generated by all the equivalent sources characterized by their ${L_{eq}}$. This illuminance can be calculated from basic photometry as:

By applying the definition of PSF in Eq. (2), Eq. (3) can be written as:

Equation (4) expresses the normalization condition for the normalization condition for the PSF as applied by the CIE.

The model presented in this paper starts from the cone-weighted spectral radiance value ${L_\alpha }$ of each pixel while the PSF is defined in photometric quantities (Eq. (2)). This inconsistency can be easily solved because the PSF is in most cases only slightly wavelength dependent [51]. Under this assumption, the PSF can equivalently be defined as the ratio of the cone-weighted spectral radiance ${L_{\alpha ,eq}}$ and irradiance values ${E_\alpha }$.

In an image-based approach, scattering can be implemented by a convolution of the original ${L_\alpha }$ map with the PSF kernel defined per pixel. A similar concept has been adopted in the field of computer graphics to render highly realistic scenes [52–55], as well as to model how images are formed in the retina [56–58]. When applying the PSF proposed by the CIE in Eq. (2) as a filter kernel, discretization and truncation is required. For our implementation, the PSF is put to zero when $\vartheta > 10^\circ $ as the change in the values outside of that range is not significant. Consequently, to keep the corneal illuminance unchanged inside the kernel, a renormalization is also required and Eqs. (3–4) can be written as:

The complete procedure to correct for stray light is as follows: for each pixel under consideration, ${E_\alpha }$ generated from the pixel is calculated and multiplied by the $PSF^{\prime}$ to calculate an intermediate image ${L_{\alpha ,eq}}$ which represents the equivalent cone weighted spectral radiance map generated by stray light from the pixel under consideration to the neighboring pixels. This is repeated for all pixels and all intermediate images are added to obtain the final image ${L_{\alpha ,eq}}$.

The effect of the convolution with $PSF^{\prime}$ is illustrated in Fig. 5 when applied on a uniform stimulus subtending 10° with ${L_{\alpha ,stim}} = 50$ and surrounded by a high luminance ring of thickness 1° and located at 6.2° off-center (a gap distance of 1.2°) and with ${L_{\alpha ,ring}} = 100$.

 

Fig. 5. The effect of convolving the input image with a PSF kernel: The top images show the full images, and the bottom images show the changes throughout a line in the full image. (a): The original input; (b) After convolution with PSF.

Download Full Size | PDF

The net effect of the convolution is to slightly decrease the original pixel values of the ring and the central circle (4% and 0.05%, respectively) and to generate some values within the gap (up to 0.8% of the original pixel values of the ring). Note that both circle and ring are generating stray light which partly compensate each other. A fixed kernel has been used over the whole image for simplicity, while strictly spoken the kernel and the normalization should be repeated for each pixel in the image. The impact of this simplification has been checked and is found to be only minor (around 2% difference).

In implementing our proposed brightness model, the correction for retinal straylight is applied by convolving ${I_\alpha }(x,y)$ with a $PSF^{\prime}$ kernel, defined with an observer age of 35 and a pigmentation factor of 0.5. No attempt was made to use an observer-specific kernel as the effect of the convolution is only minor for the experimental scenes under consideration. With the current image resolution, the kernel size width of 10° to each side of the center of the FOV (or 20° in total) corresponds to 261${\times}$ 261 pixels. The normalization factor for the $PSF^{\prime}$ was found to be equal to 0.0564. The cone excitations after straylight correction is denoted as ${L_{\alpha ,eq}}$ and the corresponding map is ${I_{\alpha ,eq}}(x,y)$.

4.3 Compression of cone responses

A non-linear compression of the cone responses is widely believed to be one of the earliest steps in visual processing [10,46,59,60]. The cone compression signals are commonly modelled with a sigmoidal curve [9,10] or using a cubic root [17]. In this step of the model, the equivalent cone excitation values are compressed using a cubic power function. This compression has shown to perform well in CAM15u [17] and avoids the issues for a complete dark pixel (${L_\alpha }$= 0) when using a log compression:

The image containing ${L_{\alpha ,c}}$ values of each pixel can be considered as a compressed cone excitation map designated as ${I_{\alpha ,c}}(x,y)$.

4.4 Receptive field response

The signal generated by the central pixel under consideration ${L_{\alpha ,c}}(x,y)$ is considered as the center signal of the receptive field, reflecting the fact that near the fovea, a one-to-one connection from the cone to the bipolar cell is assumed [61]

The surround feedback signal strength from the receptive field (representing the horizontal cell connection) is modelled as a weighted Gaussian response generated by the neighboring pixels, using a fixed standard deviation. To this extend, a Gaussian filtered image is computed by convolving ${I_{\alpha ,c}}(x,y)$ with a Gaussian kernel where the Gaussian kernel $G(x,y)$ at pixel $(x,y)$ is expressed as:

4.5 Adaptive shift and brightness output

To calculate a brightness correlate, a sigmoid function is applied as in the classical CAMs. The semi-saturation constant consists of a dark-adapted value ${\sigma _0}$ and the feedback signal strength is added as an adaptive shift, lowering the original central pixel output and modelling the inhibitory effect. This results in a brightness image ${I_Q}$ with brightness values between 0 and 1:

4.6 Determining the model parameters

The parameters included in the model are the width of the receptive field δ, the strength of the feedback from the receptive field WF, the semi-saturation constant in dark conditions ${\sigma _0}$ and the steepness of the sigmoid n. All the steps of the model have been applied for both the reference scene and the 36 test scenes. As the experimental data were collected using the method of adjustment (brightness matching), an optimization should be performed to find the optimal δ, WF and ${\sigma _0}$ value - such that the mean brightness values of the pixels belonging to the stimulus in the output image ${I_Q}$ of the test scene would be as close as possible to those of the stimulus in the reference scene. Due to this approach, the value of n cannot be determined from the experiments. Given that the choice of n does not influence the final optimization result, the value of n is chosen as n = 0.58 as suggested from CAM18sl [16].

From Fig. 2, it is observed that even for the furthest ring with the thickness of 0.33 cm, there is still an impact of the ring on the brightness of the central stimulus. For this reason, the optimization range of δ is chosen such that the width of the Gaussian kernel can cover the furthest ring; the maximum width of the kernel was taken as wide as the smaller dimension of the image. The optimization was performed using the MATLAB built-in function and the result indicates that for a δ of 151 pixels, a WF of 1.9, and a ${\sigma _0}$ of 3.0, the model gives the best approximation to all the experimental data.

The mean ${I_Q}$ values of the pixels inside the stimulus for the reference and for the test scenes using the optimal parameters are plotted in Fig. 6.

 

Fig. 6. Output of I_Q as a function of distance with regard to different ring luminance levels and ring widths

Download Full Size | PDF

Ideally, for a matching experiment, the ${I_Q}$ values of the test scenes in which gap distance, luminance and width of the ring are changing should be equal to the value of the reference scene. Overall, the model works very well, with a root mean squared error (RMSE) of 0.0018 (which counts for 0.4% of the reference stimulus brightness) indicating the root mean squared error in the brightness output of the test scene when compared to the reference scene brightness. However, for the highest ring luminance level and the closest ring distance, the model’s output generally underestimates the ring’s influence resulting in a slightly higher ${I_Q}$ value than the reference. Note that by adding the straylight correction to the model, a significant improvement in the model’s performance is found when predicting the impact of the closest ring at higher luminance levels (RMSE = 0.0029 without straylight correction). The impact of stray light will become even more important when studying scenes containing high luminance areas, which also justifies the inclusion of this step in the model. Nevertheless, this straightforward model with a limited number of free parameters is able to predict the matching experiments.

An example of the brightness output map is given in Fig. 7.

 

Fig. 7. The brightness output map: (a) The full image. (b) The value of each pixel from a line cut through the image.

Download Full Size | PDF

5. Discussion

5.1 Receptive field size

The optimized kernel standard deviation δ of 151 pixels implies that the width of the Gaussian kernel stopped at 4${\times}$δ is 604 pixels or approximately 45° in FOV. Typically, the receptive fields in the retina have a relatively small size, ranging from a few arc minutes (near the fovea) to a bit more than 10° in the periphery [62]. This does not fit with the large filter kernel size from the optimization, but the large filter size might correspond to the receptive field in the visual cortex, where the receptive field in the medial superior temporal area can reach to the size of 30°–50° [63,64]. The large receptive field width is also coherent with the conclusion from previous studies [7,8,13], where a filter with a large size ranging from a half to the full size of the input image is implemented to account for the luminance adaptation. This suggests that the process of perceiving brightness is determined by a large-scale adaptation, which considers the context of the whole environment in which the stimulus is viewed. With such a wide receptive field, the effect of involuntarily eye movements will be masked as this contribution involves blurring on a much smaller scale of typically 1° in FOV [56]. Note that the wide receptive field is also the reason that the ${I_G}$ and ${I_Q}$ values do not change very much over the stimulus area itself and no distinctive brightness jumps at the edges are observed, in line with the visual observations (Fig. 7).

From Fig. 6, a slight under-performance of the model when the rings are presented at the closest distance can be observed. This suggests that at such small distances, the impact might not simply be considered as an adaptation state but other effects such as simultaneous contrast should also be taken into consideration. Moreover, one should also consider that when the ring comes closer to the fovea, the receptive field size is also smaller near the fovea and the inhibition becomes stronger [62]

Additionally, it is noticeable from Fig. 7 that the ${I_Q}$ values for the pixels belonging to the dark gap between the ring and the stimulus are no longer completely dark and they receive a rather significant brightness value. However, the dark gap between the ring and the stimulus was not reported as luminous by the observers. Again, it appears that at the closer distance, a stronger contrast perception effect might be active, suggesting that an additional receptive field mechanism characterized by a smaller size should probably be considered.

5.2 Self-adaptation

The ${I_Q}$ value of the stimulus in the reference situation (complete dark background) is very close to 0.5, which points to the fact that the ${I_{\alpha ,c}}(x,y)$ values of the pixels of the reference are close to the sum of the dark semi-saturation value ${\sigma _0}$ and the feedback signal strength ${I_{\alpha ,G}}(x,y)$ which is only due to the stimulus itself. If one would ignore this feedback (by putting WF equal to zero), ${I_Q}$ values equal to around 0.55 would have been obtained. This illustrates the effect of adaptation to the stimulus itself.

5.3 Validation of the model

As the model is established based on rather basic and simple non-uniform backgrounds, i.e. a luminous ring in a dark area, it would be good to verify the performance of the model by applying the model to a few existing datasets from other experiments which were performed to study the perception of brightness for self-luminous stimuli. Sun et al. [24] presented a study about the influence of background luminance and background size on the brightness perception. In their study, a brightness evaluation experiment using magnitude estimation method was performed. Within the scope of the study, the background was defined as the area immediately adjacent to the stimulus and the surround is the remaining area of the screen used in the experiment, which was adjacent to the background. The experiment was set up with three different stimulus luminance levels (19, 88 and 227 cd/m²), 3 background luminance levels (0.09, 88 and 478 cd/m²), 2 surround luminance levels (0.09 and 478 cd/m²), 4 background sizes (0%, 12.5%, 50% and 100% of the screen size) and 3 background orientations (horizontal rectangle, vertical rectangle and 16:9 square).

Based on their experimental details, a set of virtual images were created with the resolution of 1280 675 pixels and a central circular stimulus with a diameter of 52 pixels corresponding to a FOV of 4° when viewed from a distance of 40 cm. As the stimuli and the backgrounds described in these papers are neutral grey, the ${L_\alpha }$ values for the input images were chosen to be equal to the luminance levels of the stimuli and the background described in the corresponding references. Based on the nature of the experiment in our study, only the situations with luminous background and dark surround are used in this evaluation. ${L_\alpha }$ of the stimulus was chosen as 19, 88 and 227, and ${L_\alpha }$ of the background was chosen as 478. The result of the simulation is shown in Fig. 8.

 

Fig. 8. The brightness output under different stimulus luminance and background sizes with a background luminance of 478 cd/m²: (Above) The psychophysics experimental result. (Below) The output of our proposed model.

Download Full Size | PDF

With a Spearman’s ranking correlation coefficient of 0.8281 when comparing the model’s output with the visual data of Sun et al., there seems to be a reasonably good agreement between the two datasets. It is observed that the model is generally capable of predicting the decrease in perceived brightness when the luminous background increases in size. The stabilization in the perceived brightness when the background size enlarges from 50% to 100% of the screen size is also predicted, however, in the original study, the most significant brightness decrease happened when the background size went from 0% to 12.5% of the screen, an effect which is less pronounced in the model prediction. This again suggests that an additional smaller receptive field-based interaction represented as a filter with a smaller standard deviation might be needed in the model, emphasizing short distance effects. According to Sun et al. [24], there is almost no effect of background orientation for all background sizes. This is confirmed by the model as long as the background is large (50% or 100% of the image). When the background size is at 12.5% of the screen size, the model predicts a brightness difference of 0.1 when changing from vertical to horizontal, and from horizontal to squared background. This is possibly due to the changes in the number of luminous pixels contributing to the kernel with changing background orientation when convolving the image with a large filter kernel.

The concept of a neutral luminous ring in a dark background can be easily extended to a neutral and uniform luminous background by increasing the width of the ring. Under these conditions, the model CAM18sl should be applied. In establishing this model, Hermans et al. [40] performed experiments in which a uniform stimulus of 10° was seen on a uniform self-luminous background. Six stimulus luminance levels were chosen as 50, 125, 250, 500, 750 and 900 cd/m² and 15 background luminance levels were chosen between 0 and 960 cd/m². The brightness of the stimulus is evaluated by magnitude estimation with respect to a reference stimulus; 20 observers participated in the experiment.

Based on the experimental details described in the paper, a set of virtual images were created with the resolution of 1280 ${\times}$ 675 pixels and a central circular stimulus with a diameter of 135 pixels corresponding to a FOV of 10° when viewed from a distance of 40 cm. The ${L_\alpha }$ values chosen for the stimulus and the background were the same values as the luminance values of the stimulus and the background in the experiment by Hermans et al. [40] . In total, 90 virtual scenes were used for this evaluation. The output of the model in relation to the background luminance was compared to the experimental brightness data Q_obs (Fig. 9). In Fig. 10, the model prediction is compared to the outcome of the visual experiment results and to the prediction of CAM18sl.

 

Fig. 9. Brightness output for the tested scenes from: (a) Experimental results of Hermans et al. [40]; (b) Brightness prediction of our model

Download Full Size | PDF

 

Fig. 10. The brightness output of the model as a function of the brightness from (a) Experimental data; (b) CAM18sl prediction The icons inside the red circles correspond to the unrelated stimuli.

Download Full Size | PDF

From Fig. 9, it can be seen that the image-based model is able to showcase the dependency of perceived brightness on stimulus and background luminance: when the background luminance increases, the brightness of the stimulus decreases, and the stimulus brightness increases with increasing stimulus luminance. The model also succeeds in predicting the sharp fall in brightness when the background goes from completely dark to 50 cd/m². From Fig. 10, the proposed model also has a good agreement with the output from the observers and from CAM18sl with the Spearman’s ranking correlation coefficients of 0.9870 and 0.9987, respectively. However, there is a slight overestimation in brightness for unrelated stimuli or, alternatively, a systematic underestimation of the scenes with a luminous background. It is worth noting that the model is developed based on rather small luminous background areas; applying the model to scenes with a very large background size, as is the case for the Hermans et al. data, is quite challenging.

It is also observed that there is no linear relationship between the output of our model and that of CAM18sl, which is believed to be the result of 2-step compression used in our model, while CAM18sl is only using a single compression step.

6. Conclusion

Various CAMs have been developed to predict how humans perceive colors from the optical input of the stimuli. However, with the traditional CAMs, the applications are still limited to a uniform stimulus and background [16]. To extend the applications of such CAMs to non-uniform backgrounds, image-based CAMs have been created, yet, there are still some shortcomings when they are applied to self-luminous scenes [14]. This leads to the need of a comprehensive image-based color appearance model which can overcome the limitations of current CAMs and image CAMs when working with self-luminous scenes; such a model could be called a Lighting Appearance Model (LAM). To move towards developing a LAM, we believe that the first step is to create a comprehensive brightness model for non-uniform backgrounds including self-adaptation.

In this paper, a series of visual brightness experiments have been conducted to study the impact of introducing a luminous ring-shaped area to a dark background on the brightness perception of a central stimulus. Three parameters of the luminous ring have been studied, including the distance from the central stimulus to the ring, the thickness of the ring and the luminance of the ring. In line with various studies, it is clearly shown that by adding a luminous ring to the scene, the brightness of the central stimulus decreases substantially, even for the smallest thickness (0.33 cm) and the largest stimulus-to-ring distance (16.1°). This phenomenon appears to be stronger when the ring is closer to the stimulus and the impact also increases with the increasing luminance of the ring. The study also confirmed the area effect [24], which was already reported in the literature: as the area of the luminous ring increases, the target stimulus appears to be darker.

An image-based model to simulate the observed phenomenon is proposed, which is highly inspired by the basic physiology of the retina. The model includes cube root compression, scattering in the eye, a receptive field concept, inhibition by neighboring pixels and sigmoid compression. The model has been applied to the experimental brightness data and three parameters are optimized: the width of a Gaussian kernel mimicking the surround signal of the receptive field, the overall weighting factor representing the inhibition strength and the semi-saturation constant for a dark-adapted environment. A standard deviation of 151 pixels representing a receptive field with the coverage up to 45° and a weighting factor of 1.9, shows the best performance of the model. The large receptive field width suggests that the brightness perception and the adaptation to the ring might be the result of the processing at the later stage of the visual pathway such as in the visual cortex, where the receptive field has a large size up to 50° [63,64]. This large filter size is also in line with previous findings [7,8,13], which supports the idea that the adaptation is mainly linked to the global context where the stimulus is viewed. The model provides a generally good performance in predicting the effect of the area, the distance and the intensity of the ring, though there is some underestimation of the effects with the closest ring distance, which suggests the need of having an additional but smaller receptive field mechanism in the model to simulate more local effects.

Additionally, the performance of the model is evaluated using self-luminous scenes created based on previous studies about brightness for self-luminous stimuli [24,40]. The results show that the model can predict the perceived brightness behavior of those studies, illustrating its robustness. As such, the proposed image-based model appears promising to deal with non-uniform stimuli and complex scenes and can be considered as an important step in search of a generic LAM.

Future work will concentrate on considering adapting the size and the weighting factor of the Gaussian kernel applied to both the center and the surround signal, according to the retinal position, to consider both short- and long-range receptive field sizes, to include a larger range of luminance levels and to include colored stimuli and backgrounds.