Abstract

Sensor sharpening [J. Opt. Soc. Am. A 11, 1553 (1994)] has been proposed as a method for improving computational color constancy, but it has not been thoroughly tested in practice with existing color constancy algorithms. In this paper we study sensor sharpening in the context of viable color constancy processing, both theoretically and empirically, and on four different cameras. Our experimental findings lead us to propose a new sharpening method that optimizes an objective function that includes terms that minimize negative sensor responses as well as the sharpening error for multiple illuminants instead of a single illuminant. Further experiments suggest that this method is more effective for use with several known color constancy algorithms.

© 2001 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. G. D. Finlayson, M. S. Drew, B. V. Funt, “Spectral sharpening: sensor transformations for improved color constancy,” J. Opt. Soc. Am. A 11, 1553–1563 (1994).
    [CrossRef]
  2. K. Barnard, B. Funt, “Experiments in sensor sharpening for color constancy,” in Proceedings of the IS&T/SID Sixth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1998), pp. 43–46.
  3. K. Barnard, B. Funt, “Camera calibration for color research,” Color Res. Appl. (to be published).
  4. P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Digital color cameras. 2. Spectral response,” , available from http://www.hpl.hp.com/techreports/97/HPL-97-54.html (1997).
  5. D. Forsyth, “A novel algorithm for color constancy,” Int. J. Comput. Vision 5, 5–36 (1990).
    [CrossRef]
  6. G. D. Finlayson, M. S. Drew, B. V. Funt, “Color constancy: generalized diagonal transforms suffice,” J. Opt. Soc. Am. A 11, 3011–3020 (1994).
    [CrossRef]
  7. Available from http://www.cs.sfu.ca/~colour/data .
  8. K. Barnard, L. Martin, B. Funt, A. Coath, “Data for colour research,” Color Res. Appl. (to be published).
  9. G. D. Finlayson, “Color in perspective,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 1034–1038 (1996).
    [CrossRef]
  10. K. Barnard, “Computational colour constancy: taking theory into practice,” M.Sc. thesis (Simon Fraser University, School of Computing Burnaby, B.C., Canada), available from ftp://fas.sfu.ca/pub/cs/theses/1995/KobusBarnardMSc.ps.gz (1995).
  11. G. Finlayson, S. Hordley, “A theory of selection for gamut mapping colour constancy,” Image Vision Comput. 17, 545–588 (1999).
    [CrossRef]
  12. K. Barnard, “Practical colour constancy,” (Ph.D. thesis Simon Fraser University, School of Computing, Burnaby, B.C., Canada), available from ftp://fas.sfu.ca/pub/cs/theses/1999/KobusBarnardPhD.ps.gz (1999).
  13. K. Barnard, V. Cardei, B. Funt, “A comparison of computational color constancy algorithms. Part one. Methodology and experiments with synthesized data,” available from http://www.cs.berkeley.edu/~kobus/research/publications/comparison_1 .
  14. K. Barnard, L. Martin, A. Coath, B. Funt, “A comparison of color constancy algorithms. Part two. Experiments with image data,” available from http://www.cs.berkeley.edu/~kobus/research/publications/comparison_2 .
  15. For example, in the preliminary study, the results indicated that the two sharpening methods tested (“ave” and “opt”) yield a modest benefit in conjunction with the SCALE-BY-MAXalgorithm (labeled “Retinex” in that paper). However, the SCALE-BY-MAXalgorithm was helped by the removal of data rejected by the gamut-mapping algorithms. Without this help, the conclusion is reversed in several circumstances.
  16. G. Finlayson, M. Drew, “Positive Bradford curves through sharpening,” in Proceedings of the IS&T/SID Seventh Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 227–232.
  17. B. Funt, K. Barnard, L. Martin, “Is colour constancy good enough?” in Proceedings of the 5th European Conference on Computer Vision (Springer, Berlin, 1998), pp. I:445–459.
  18. J. J. McCann, S. P. McKee, T. H. Taylor, “Quantitative studies in Retinex theory,” Vision Res. 16, 445–458 (1976).
    [CrossRef]
  19. E. H. Land, “The Retinex theory of color vision,” Sci. Am. 237, 108–129 (1977).
    [CrossRef] [PubMed]
  20. B. K. P. Horn, “Determining lightness from an image,” Comput. Vision Graph. Image Process. 3, 277–299 (1974).
    [CrossRef]
  21. B. Funt, V. Cardei, K. Barnard, “Learning color constancy,” in Proceedings of the IS&T/SID Fourth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 58–60.
  22. G. D. Finlayson, P. H. Hubel, S. Hordley, “Color by correlation,” in Proceedings of the IS&T/SID Fifth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1997), pp. 6–11.
  23. M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).
  24. E. L. Krinov, Spectral Reflectance Properties of Natural Formations (National Research Council of Canada, Ottawa, 1947).

1999

G. Finlayson, S. Hordley, “A theory of selection for gamut mapping colour constancy,” Image Vision Comput. 17, 545–588 (1999).
[CrossRef]

1996

G. D. Finlayson, “Color in perspective,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 1034–1038 (1996).
[CrossRef]

1994

1990

D. Forsyth, “A novel algorithm for color constancy,” Int. J. Comput. Vision 5, 5–36 (1990).
[CrossRef]

1977

E. H. Land, “The Retinex theory of color vision,” Sci. Am. 237, 108–129 (1977).
[CrossRef] [PubMed]

1976

J. J. McCann, S. P. McKee, T. H. Taylor, “Quantitative studies in Retinex theory,” Vision Res. 16, 445–458 (1976).
[CrossRef]

1974

B. K. P. Horn, “Determining lightness from an image,” Comput. Vision Graph. Image Process. 3, 277–299 (1974).
[CrossRef]

Barnard, K.

K. Barnard, B. Funt, “Experiments in sensor sharpening for color constancy,” in Proceedings of the IS&T/SID Sixth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1998), pp. 43–46.

K. Barnard, B. Funt, “Camera calibration for color research,” Color Res. Appl. (to be published).

K. Barnard, L. Martin, B. Funt, A. Coath, “Data for colour research,” Color Res. Appl. (to be published).

K. Barnard, “Computational colour constancy: taking theory into practice,” M.Sc. thesis (Simon Fraser University, School of Computing Burnaby, B.C., Canada), available from ftp://fas.sfu.ca/pub/cs/theses/1995/KobusBarnardMSc.ps.gz (1995).

K. Barnard, “Practical colour constancy,” (Ph.D. thesis Simon Fraser University, School of Computing, Burnaby, B.C., Canada), available from ftp://fas.sfu.ca/pub/cs/theses/1999/KobusBarnardPhD.ps.gz (1999).

B. Funt, K. Barnard, L. Martin, “Is colour constancy good enough?” in Proceedings of the 5th European Conference on Computer Vision (Springer, Berlin, 1998), pp. I:445–459.

B. Funt, V. Cardei, K. Barnard, “Learning color constancy,” in Proceedings of the IS&T/SID Fourth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 58–60.

Brainard, D. H.

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Digital color cameras. 2. Spectral response,” , available from http://www.hpl.hp.com/techreports/97/HPL-97-54.html (1997).

Cardei, V.

B. Funt, V. Cardei, K. Barnard, “Learning color constancy,” in Proceedings of the IS&T/SID Fourth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 58–60.

Coath, A.

K. Barnard, L. Martin, B. Funt, A. Coath, “Data for colour research,” Color Res. Appl. (to be published).

Drew, M.

G. Finlayson, M. Drew, “Positive Bradford curves through sharpening,” in Proceedings of the IS&T/SID Seventh Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 227–232.

Drew, M. S.

Farrell, J. E.

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Digital color cameras. 2. Spectral response,” , available from http://www.hpl.hp.com/techreports/97/HPL-97-54.html (1997).

Finlayson, G.

G. Finlayson, S. Hordley, “A theory of selection for gamut mapping colour constancy,” Image Vision Comput. 17, 545–588 (1999).
[CrossRef]

G. Finlayson, M. Drew, “Positive Bradford curves through sharpening,” in Proceedings of the IS&T/SID Seventh Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 227–232.

Finlayson, G. D.

G. D. Finlayson, “Color in perspective,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 1034–1038 (1996).
[CrossRef]

G. D. Finlayson, M. S. Drew, B. V. Funt, “Spectral sharpening: sensor transformations for improved color constancy,” J. Opt. Soc. Am. A 11, 1553–1563 (1994).
[CrossRef]

G. D. Finlayson, M. S. Drew, B. V. Funt, “Color constancy: generalized diagonal transforms suffice,” J. Opt. Soc. Am. A 11, 3011–3020 (1994).
[CrossRef]

G. D. Finlayson, P. H. Hubel, S. Hordley, “Color by correlation,” in Proceedings of the IS&T/SID Fifth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1997), pp. 6–11.

Forsyth, D.

D. Forsyth, “A novel algorithm for color constancy,” Int. J. Comput. Vision 5, 5–36 (1990).
[CrossRef]

Funt, B.

K. Barnard, B. Funt, “Camera calibration for color research,” Color Res. Appl. (to be published).

K. Barnard, B. Funt, “Experiments in sensor sharpening for color constancy,” in Proceedings of the IS&T/SID Sixth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1998), pp. 43–46.

B. Funt, K. Barnard, L. Martin, “Is colour constancy good enough?” in Proceedings of the 5th European Conference on Computer Vision (Springer, Berlin, 1998), pp. I:445–459.

K. Barnard, L. Martin, B. Funt, A. Coath, “Data for colour research,” Color Res. Appl. (to be published).

B. Funt, V. Cardei, K. Barnard, “Learning color constancy,” in Proceedings of the IS&T/SID Fourth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 58–60.

Funt, B. V.

Gershon, R.

M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).

Hordley, S.

G. Finlayson, S. Hordley, “A theory of selection for gamut mapping colour constancy,” Image Vision Comput. 17, 545–588 (1999).
[CrossRef]

G. D. Finlayson, P. H. Hubel, S. Hordley, “Color by correlation,” in Proceedings of the IS&T/SID Fifth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1997), pp. 6–11.

Horn, B. K. P.

B. K. P. Horn, “Determining lightness from an image,” Comput. Vision Graph. Image Process. 3, 277–299 (1974).
[CrossRef]

Hubel, P. H.

G. D. Finlayson, P. H. Hubel, S. Hordley, “Color by correlation,” in Proceedings of the IS&T/SID Fifth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1997), pp. 6–11.

Iwan, L. S.

M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).

Krinov, E. L.

E. L. Krinov, Spectral Reflectance Properties of Natural Formations (National Research Council of Canada, Ottawa, 1947).

Land, E. H.

E. H. Land, “The Retinex theory of color vision,” Sci. Am. 237, 108–129 (1977).
[CrossRef] [PubMed]

Martin, L.

B. Funt, K. Barnard, L. Martin, “Is colour constancy good enough?” in Proceedings of the 5th European Conference on Computer Vision (Springer, Berlin, 1998), pp. I:445–459.

K. Barnard, L. Martin, B. Funt, A. Coath, “Data for colour research,” Color Res. Appl. (to be published).

McCann, J. J.

J. J. McCann, S. P. McKee, T. H. Taylor, “Quantitative studies in Retinex theory,” Vision Res. 16, 445–458 (1976).
[CrossRef]

McKee, S. P.

J. J. McCann, S. P. McKee, T. H. Taylor, “Quantitative studies in Retinex theory,” Vision Res. 16, 445–458 (1976).
[CrossRef]

Taylor, T. H.

J. J. McCann, S. P. McKee, T. H. Taylor, “Quantitative studies in Retinex theory,” Vision Res. 16, 445–458 (1976).
[CrossRef]

Tietz, J. D.

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Digital color cameras. 2. Spectral response,” , available from http://www.hpl.hp.com/techreports/97/HPL-97-54.html (1997).

Vora, P. L.

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Digital color cameras. 2. Spectral response,” , available from http://www.hpl.hp.com/techreports/97/HPL-97-54.html (1997).

Vrhel, M. J.

M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).

Color Res. Appl.

M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).

Comput. Vision Graph. Image Process.

B. K. P. Horn, “Determining lightness from an image,” Comput. Vision Graph. Image Process. 3, 277–299 (1974).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell.

G. D. Finlayson, “Color in perspective,” IEEE Trans. Pattern Anal. Mach. Intell. 18, 1034–1038 (1996).
[CrossRef]

Image Vision Comput.

G. Finlayson, S. Hordley, “A theory of selection for gamut mapping colour constancy,” Image Vision Comput. 17, 545–588 (1999).
[CrossRef]

Int. J. Comput. Vision

D. Forsyth, “A novel algorithm for color constancy,” Int. J. Comput. Vision 5, 5–36 (1990).
[CrossRef]

J. Opt. Soc. Am. A

Sci. Am.

E. H. Land, “The Retinex theory of color vision,” Sci. Am. 237, 108–129 (1977).
[CrossRef] [PubMed]

Vision Res.

J. J. McCann, S. P. McKee, T. H. Taylor, “Quantitative studies in Retinex theory,” Vision Res. 16, 445–458 (1976).
[CrossRef]

Other

K. Barnard, “Computational colour constancy: taking theory into practice,” M.Sc. thesis (Simon Fraser University, School of Computing Burnaby, B.C., Canada), available from ftp://fas.sfu.ca/pub/cs/theses/1995/KobusBarnardMSc.ps.gz (1995).

K. Barnard, “Practical colour constancy,” (Ph.D. thesis Simon Fraser University, School of Computing, Burnaby, B.C., Canada), available from ftp://fas.sfu.ca/pub/cs/theses/1999/KobusBarnardPhD.ps.gz (1999).

K. Barnard, V. Cardei, B. Funt, “A comparison of computational color constancy algorithms. Part one. Methodology and experiments with synthesized data,” available from http://www.cs.berkeley.edu/~kobus/research/publications/comparison_1 .

K. Barnard, L. Martin, A. Coath, B. Funt, “A comparison of color constancy algorithms. Part two. Experiments with image data,” available from http://www.cs.berkeley.edu/~kobus/research/publications/comparison_2 .

For example, in the preliminary study, the results indicated that the two sharpening methods tested (“ave” and “opt”) yield a modest benefit in conjunction with the SCALE-BY-MAXalgorithm (labeled “Retinex” in that paper). However, the SCALE-BY-MAXalgorithm was helped by the removal of data rejected by the gamut-mapping algorithms. Without this help, the conclusion is reversed in several circumstances.

G. Finlayson, M. Drew, “Positive Bradford curves through sharpening,” in Proceedings of the IS&T/SID Seventh Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 227–232.

B. Funt, K. Barnard, L. Martin, “Is colour constancy good enough?” in Proceedings of the 5th European Conference on Computer Vision (Springer, Berlin, 1998), pp. I:445–459.

K. Barnard, B. Funt, “Experiments in sensor sharpening for color constancy,” in Proceedings of the IS&T/SID Sixth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1998), pp. 43–46.

K. Barnard, B. Funt, “Camera calibration for color research,” Color Res. Appl. (to be published).

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Digital color cameras. 2. Spectral response,” , available from http://www.hpl.hp.com/techreports/97/HPL-97-54.html (1997).

Available from http://www.cs.sfu.ca/~colour/data .

K. Barnard, L. Martin, B. Funt, A. Coath, “Data for colour research,” Color Res. Appl. (to be published).

B. Funt, V. Cardei, K. Barnard, “Learning color constancy,” in Proceedings of the IS&T/SID Fourth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 58–60.

G. D. Finlayson, P. H. Hubel, S. Hordley, “Color by correlation,” in Proceedings of the IS&T/SID Fifth Color Imaging Conference: Color Science, Systems and Applications (The Society for Imaging Science and Technology, Springfield, Va., 1997), pp. 6–11.

E. L. Krinov, Spectral Reflectance Properties of Natural Formations (National Research Council of Canada, Ottawa, 1947).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (2)

Fig. 1
Fig. 1

Original camera sensors, sensors corresponding to sharpening based on the average of the database of illuminants, and sensors corresponding to the multiple illuminant with positivity sharpening method introduced in this paper for (a) the Sony DXC-930, (b) the Kodak DCS-460, (c) the Kodak DCS-200, and (d) the Kodak DCS-420. Note that for the DXC-930 [(a)], the sensors sharpened with the multiple-illuminant-with-positivity method are very close to the original ones, and the two sets of sensor curves are blended for much of the wavelength range. Since these sensors are already “sharp,” this is encouraging. The three Kodak cameras have blue sensors that have secondary peaks in the red region of the spectrum as well as significant responses in the green region. The red sensors also have significant response in the green region. Sharpening essentially removes these characteristics and also yields narrower green sensors. Thus for these three cameras, the term “sharpening” is very appropriate.

Fig. 2
Fig. 2

Rms RGB mapping error between the corrected image and the target image by algorithm-sharpening method combination for (a) the Sony DXC-930, (b) the Kodak DCS-460, (c) the Kodak DCS-200, and (d) the Kodak DCS-420. Note that the scale would have to be significantly larger to illustrate fully the extent of all the bars [7× in (a), 14× in (b), 30× in (c), and 20× in (d)]. These extreme errors are due to instabilities in the computation that occur with small or negative components. Their exact values depend on the test set. Therefore the results should be taken in a qualitative sense. RMS corresponds to rms in the text.

Tables (5)

Tables Icon

Table 1 Sharpening Results for the Sony DXC-930 Video Camera on Synthetic Dataa

Tables Icon

Table 2 Sharpening Results for the Kodak DCS-460 Digital Camera on Synthetic Dataa

Tables Icon

Table 3 Sharpening Results for the Kodak DCS-200 Digital Camera on Synthetic Dataa

Tables Icon

Table 4 Sharpening Results for the Kodak DCS-420 Digital Camera on Synthetic Dataa

Tables Icon

Table 5 Sharpening Results for the Kodak DCS-460 Digital Camera on Image Data for the Incandescent Illuminant Groupa

Equations (12)

Equations on this page are rendered with MathJax. Learn more.

(r, g, b)=(r, g, b)diag(r/r, g/g, b/b).
E=iAiT DiT -1-BF,
f(x)=(x-offset)2if(x<offset)0otherwise.
N=[trace(T T)-3]2.
E+λPP+λNN,
w=1n inridiag(wc./gc),
w=1n inriTdiag[(wcT)./(gcT)]T-1.
D=diaggc./1n inri.
D=diag(gcT)./1n inriT.
D=diag(wc./w)
D=diag[(wcT)./(wT)].
T=10-13010001.

Metrics