Abstract

Pixel saturation, in which the incident light at a pixel causes one of the color channels of the camera sensor to respond at its maximum value, can produce undesirable artifacts in digital color images. We present a Bayesian algorithm that estimates what the saturated channel’s value would have been in the absence of saturation. The algorithm uses the nonsaturated responses from the other color channels, together with a multivariate normal prior that captures the correlation in response across color channels. The prior may be estimated directly from the image data, since most image pixels are not saturated. Given the prior and the responses of the nonsaturated channels, the algorithm returns the optimal expected mean square estimate for the true response. Extensions of the algorithm to the case in which more than one channel is saturated are also discussed. Both simulations and examples with real images are presented to show that the algorithm is effective.

© 2004 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. P. Longere, D. H. Brainard, “Simulation of digital camera images from hyperspectral input,” in Vision Models and Applications to Image and Video Processing, C. van den Branden Lambrecht, ed. (Kluwer Academic, Boston, Mass., 2001), pp. 123–150.
  2. J. Holm, I. Tastl, L. Hanlon, P. Hubel, “Color processing for digital photography,” in Colour Engineering: Achieving Device Independent Colour, P. Green, L. MacDonald, eds. (Wiley, New York, 2002), pp. 179–217.
  3. J. Holm, “A strategy for pictorial digital image processing,” in Proceedings of the IS&T/SID 4th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 194–201.
  4. R. Kimmel, “Demosaicing: image reconstruction from color CCD samples,” IEEE Trans. Image Process. 8, 1221–1228 (1999).
    [CrossRef]
  5. R. Kakarala, Z. Baharav, “Adaptive demosaicing with the principal vector method,” IEEE Trans. Consum. Electron. 48, 932–937 (2002).
    [CrossRef]
  6. D. H. Brainard, D. Sherman, “Reconstructing images from trichromatic samples: from basic research to practical applications,” in Proceedings of the 3rd IS&T/SID Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1995), pp. 4–10.
  7. B. Tao, I. Tastl, T. Cooper, M. Blasgen, E. Edwards, “Demosaicing using human visual properties and wavelet interpolation filtering,” in Proceedings of the IS&T/SID 7th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 252–256.
  8. J. A. S. Viggiano, “Minimal-knowledge assumptions in digital still camera characterization. I: Uniform distribution, Toeplitz correlation,” in Proceedings of the IS&T/SID 9th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 2001), pp. 332–336.
  9. D. H. Brainard, Colorimetry (McGraw-Hill, New York, 1995), pp. 26.21–26.54.
  10. J. Holm, “Photographic tone and colour reproduction goals,” in Proceedings of the CIE Expert Symposium on Colour Standards for Image Technology (Bureau Central de la CIE, Vienna, 1996), pp. 51–56.
  11. G. W. Larson, H. Rushmeier, C. Piatko, “A visibility matching tone reproduction operator for high dynamic range scenes,” IEEE Trans. Visualization Comput. Graph. 3, 291–306 (1997).
    [CrossRef]
  12. G. D. Finlayson, M. S. Drew, “The maximum ignorance assumption with positivity,” in Proceedings of the IS&T/SID 4th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 202–204.
  13. D. H. Brainard, W. T. Freeman, “Bayesian color constancy,” J. Opt. Soc. Am. A 14, 1393–1411 (1997).
    [CrossRef]
  14. E. P. Simoncelli, “Bayesian denoising of visual images in the wavelet domain,” in Bayesian Inference in Wavelet Based Models, P. Müller, B. Vidakovic, eds., Vol. 141 of Lecture Notes in Statistics (Springer-Verlag, New York, 1999), pp. 291–308.
    [CrossRef]
  15. Y. Weiss, E. H. Adelson, “Slow and smooth: a Bayesian theory for the combination of local motion signals in human vision,” (MIT, Cambridge, Mass., 1998).
  16. X. Zhang, D. H. Brainard, “Method and apparatus for estimating true color values for saturated color values in digitally captured image data,” U.S. patent6,731,794 (May4, 2004).
  17. T. O. Berger, Statistical Decision Theory and Bayesian Analysis (Springer-Verlag, New York, 1985).
  18. P. M. Lee, Bayesian Statistics (Oxford U. Press, London, 1989).
  19. D. H. Brainard, “Bayesian method for reconstructing color images from trichromatic samples,” in Proceedings of the IS&T 47th Annual Meeting (Society for Imaging Science and Technology, Springfield, Va., 1994), pp. 375–380.
  20. W. R. Dillon, M. Goldstein, Multivariate Analysis (Wiley, New York, 1984).
  21. X. Zhang, D. H. Brainard, “Bayes color correction method for non-colorimetric digital image sensors,” in Proceedings of the IS&T/SID 12th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 2004).
  22. J. E. Adams, J. F. Hamilton, “Adaptive color plane interpolation in single sensor color electronic camera,” U.S. patent5652621 (July29, 1997).
  23. H. J. Trussell, R. E. Hartwig, “Mathematics for demosaicing,” IEEE Trans. Image Process. 11, 485–492 (2002).
    [CrossRef]
  24. M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).
  25. P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Image capture: simulation of sensor responses from hyperspectral images,” IEEE Trans. Image Process. 10, 307–316 (2001).
    [CrossRef]
  26. CIE, “Industrial colour-difference evaluation,” (Bureau Central de la CIE, Vienna, 1995).
  27. S. Pattanaik, J. Ferwerda, M. Fairchild, D. Greenberg, “A multiscale model of adaptation and spatial vision for realistic image display,” in Proceedings of SIGGRAPH’98 ( www.siggraph.org , 1998), pp. 287–298.
  28. F. Xiao, J. M. DiCarlo, P. B. Catrysse, B. A. Wandell, “High dynamic range imaging of natural scenes,” in Final Program and Proceedings of the 10th IS&T/SID Color Imaging Conference: Color Science, Systems and Applications (Society for Imaging Science and Technology, Springfield, Va., 2002), pp. 337–342.
  29. D. Yang, B. Fowler, “A 640×512 CMOS image sensor with ultrawide dynamic range floating-point pixel-level ADC,” IEEE J. Solid-State Circuits 34, 1821–1834 (1999).
    [CrossRef]
  30. S. K. Nayar, T. Mitsunaga, “High dynamic range imaging: spatially varying pixel exposures,” in Proceedings of IEEE CVPR (IEEE Press, Piscataway, N.J., 2000), pp. 1472–1479.

2002 (2)

R. Kakarala, Z. Baharav, “Adaptive demosaicing with the principal vector method,” IEEE Trans. Consum. Electron. 48, 932–937 (2002).
[CrossRef]

H. J. Trussell, R. E. Hartwig, “Mathematics for demosaicing,” IEEE Trans. Image Process. 11, 485–492 (2002).
[CrossRef]

2001 (1)

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Image capture: simulation of sensor responses from hyperspectral images,” IEEE Trans. Image Process. 10, 307–316 (2001).
[CrossRef]

1999 (2)

D. Yang, B. Fowler, “A 640×512 CMOS image sensor with ultrawide dynamic range floating-point pixel-level ADC,” IEEE J. Solid-State Circuits 34, 1821–1834 (1999).
[CrossRef]

R. Kimmel, “Demosaicing: image reconstruction from color CCD samples,” IEEE Trans. Image Process. 8, 1221–1228 (1999).
[CrossRef]

1997 (2)

D. H. Brainard, W. T. Freeman, “Bayesian color constancy,” J. Opt. Soc. Am. A 14, 1393–1411 (1997).
[CrossRef]

G. W. Larson, H. Rushmeier, C. Piatko, “A visibility matching tone reproduction operator for high dynamic range scenes,” IEEE Trans. Visualization Comput. Graph. 3, 291–306 (1997).
[CrossRef]

1994 (1)

M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).

Adams, J. E.

J. E. Adams, J. F. Hamilton, “Adaptive color plane interpolation in single sensor color electronic camera,” U.S. patent5652621 (July29, 1997).

Adelson, E. H.

Y. Weiss, E. H. Adelson, “Slow and smooth: a Bayesian theory for the combination of local motion signals in human vision,” (MIT, Cambridge, Mass., 1998).

Baharav, Z.

R. Kakarala, Z. Baharav, “Adaptive demosaicing with the principal vector method,” IEEE Trans. Consum. Electron. 48, 932–937 (2002).
[CrossRef]

Berger, T. O.

T. O. Berger, Statistical Decision Theory and Bayesian Analysis (Springer-Verlag, New York, 1985).

Blasgen, M.

B. Tao, I. Tastl, T. Cooper, M. Blasgen, E. Edwards, “Demosaicing using human visual properties and wavelet interpolation filtering,” in Proceedings of the IS&T/SID 7th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 252–256.

Brainard, D. H.

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Image capture: simulation of sensor responses from hyperspectral images,” IEEE Trans. Image Process. 10, 307–316 (2001).
[CrossRef]

D. H. Brainard, W. T. Freeman, “Bayesian color constancy,” J. Opt. Soc. Am. A 14, 1393–1411 (1997).
[CrossRef]

X. Zhang, D. H. Brainard, “Method and apparatus for estimating true color values for saturated color values in digitally captured image data,” U.S. patent6,731,794 (May4, 2004).

D. H. Brainard, Colorimetry (McGraw-Hill, New York, 1995), pp. 26.21–26.54.

P. Longere, D. H. Brainard, “Simulation of digital camera images from hyperspectral input,” in Vision Models and Applications to Image and Video Processing, C. van den Branden Lambrecht, ed. (Kluwer Academic, Boston, Mass., 2001), pp. 123–150.

D. H. Brainard, “Bayesian method for reconstructing color images from trichromatic samples,” in Proceedings of the IS&T 47th Annual Meeting (Society for Imaging Science and Technology, Springfield, Va., 1994), pp. 375–380.

D. H. Brainard, D. Sherman, “Reconstructing images from trichromatic samples: from basic research to practical applications,” in Proceedings of the 3rd IS&T/SID Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1995), pp. 4–10.

X. Zhang, D. H. Brainard, “Bayes color correction method for non-colorimetric digital image sensors,” in Proceedings of the IS&T/SID 12th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 2004).

Catrysse, P. B.

F. Xiao, J. M. DiCarlo, P. B. Catrysse, B. A. Wandell, “High dynamic range imaging of natural scenes,” in Final Program and Proceedings of the 10th IS&T/SID Color Imaging Conference: Color Science, Systems and Applications (Society for Imaging Science and Technology, Springfield, Va., 2002), pp. 337–342.

Cooper, T.

B. Tao, I. Tastl, T. Cooper, M. Blasgen, E. Edwards, “Demosaicing using human visual properties and wavelet interpolation filtering,” in Proceedings of the IS&T/SID 7th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 252–256.

DiCarlo, J. M.

F. Xiao, J. M. DiCarlo, P. B. Catrysse, B. A. Wandell, “High dynamic range imaging of natural scenes,” in Final Program and Proceedings of the 10th IS&T/SID Color Imaging Conference: Color Science, Systems and Applications (Society for Imaging Science and Technology, Springfield, Va., 2002), pp. 337–342.

Dillon, W. R.

W. R. Dillon, M. Goldstein, Multivariate Analysis (Wiley, New York, 1984).

Drew, M. S.

G. D. Finlayson, M. S. Drew, “The maximum ignorance assumption with positivity,” in Proceedings of the IS&T/SID 4th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 202–204.

Edwards, E.

B. Tao, I. Tastl, T. Cooper, M. Blasgen, E. Edwards, “Demosaicing using human visual properties and wavelet interpolation filtering,” in Proceedings of the IS&T/SID 7th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 252–256.

Farrell, J. E.

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Image capture: simulation of sensor responses from hyperspectral images,” IEEE Trans. Image Process. 10, 307–316 (2001).
[CrossRef]

Finlayson, G. D.

G. D. Finlayson, M. S. Drew, “The maximum ignorance assumption with positivity,” in Proceedings of the IS&T/SID 4th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 202–204.

Fowler, B.

D. Yang, B. Fowler, “A 640×512 CMOS image sensor with ultrawide dynamic range floating-point pixel-level ADC,” IEEE J. Solid-State Circuits 34, 1821–1834 (1999).
[CrossRef]

Freeman, W. T.

Gershon, R.

M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).

Goldstein, M.

W. R. Dillon, M. Goldstein, Multivariate Analysis (Wiley, New York, 1984).

Hamilton, J. F.

J. E. Adams, J. F. Hamilton, “Adaptive color plane interpolation in single sensor color electronic camera,” U.S. patent5652621 (July29, 1997).

Hanlon, L.

J. Holm, I. Tastl, L. Hanlon, P. Hubel, “Color processing for digital photography,” in Colour Engineering: Achieving Device Independent Colour, P. Green, L. MacDonald, eds. (Wiley, New York, 2002), pp. 179–217.

Hartwig, R. E.

H. J. Trussell, R. E. Hartwig, “Mathematics for demosaicing,” IEEE Trans. Image Process. 11, 485–492 (2002).
[CrossRef]

Holm, J.

J. Holm, “Photographic tone and colour reproduction goals,” in Proceedings of the CIE Expert Symposium on Colour Standards for Image Technology (Bureau Central de la CIE, Vienna, 1996), pp. 51–56.

J. Holm, I. Tastl, L. Hanlon, P. Hubel, “Color processing for digital photography,” in Colour Engineering: Achieving Device Independent Colour, P. Green, L. MacDonald, eds. (Wiley, New York, 2002), pp. 179–217.

J. Holm, “A strategy for pictorial digital image processing,” in Proceedings of the IS&T/SID 4th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 194–201.

Hubel, P.

J. Holm, I. Tastl, L. Hanlon, P. Hubel, “Color processing for digital photography,” in Colour Engineering: Achieving Device Independent Colour, P. Green, L. MacDonald, eds. (Wiley, New York, 2002), pp. 179–217.

Iwan, L. S.

M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).

Kakarala, R.

R. Kakarala, Z. Baharav, “Adaptive demosaicing with the principal vector method,” IEEE Trans. Consum. Electron. 48, 932–937 (2002).
[CrossRef]

Kimmel, R.

R. Kimmel, “Demosaicing: image reconstruction from color CCD samples,” IEEE Trans. Image Process. 8, 1221–1228 (1999).
[CrossRef]

Larson, G. W.

G. W. Larson, H. Rushmeier, C. Piatko, “A visibility matching tone reproduction operator for high dynamic range scenes,” IEEE Trans. Visualization Comput. Graph. 3, 291–306 (1997).
[CrossRef]

Lee, P. M.

P. M. Lee, Bayesian Statistics (Oxford U. Press, London, 1989).

Longere, P.

P. Longere, D. H. Brainard, “Simulation of digital camera images from hyperspectral input,” in Vision Models and Applications to Image and Video Processing, C. van den Branden Lambrecht, ed. (Kluwer Academic, Boston, Mass., 2001), pp. 123–150.

Mitsunaga, T.

S. K. Nayar, T. Mitsunaga, “High dynamic range imaging: spatially varying pixel exposures,” in Proceedings of IEEE CVPR (IEEE Press, Piscataway, N.J., 2000), pp. 1472–1479.

Nayar, S. K.

S. K. Nayar, T. Mitsunaga, “High dynamic range imaging: spatially varying pixel exposures,” in Proceedings of IEEE CVPR (IEEE Press, Piscataway, N.J., 2000), pp. 1472–1479.

Piatko, C.

G. W. Larson, H. Rushmeier, C. Piatko, “A visibility matching tone reproduction operator for high dynamic range scenes,” IEEE Trans. Visualization Comput. Graph. 3, 291–306 (1997).
[CrossRef]

Rushmeier, H.

G. W. Larson, H. Rushmeier, C. Piatko, “A visibility matching tone reproduction operator for high dynamic range scenes,” IEEE Trans. Visualization Comput. Graph. 3, 291–306 (1997).
[CrossRef]

Sherman, D.

D. H. Brainard, D. Sherman, “Reconstructing images from trichromatic samples: from basic research to practical applications,” in Proceedings of the 3rd IS&T/SID Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1995), pp. 4–10.

Simoncelli, E. P.

E. P. Simoncelli, “Bayesian denoising of visual images in the wavelet domain,” in Bayesian Inference in Wavelet Based Models, P. Müller, B. Vidakovic, eds., Vol. 141 of Lecture Notes in Statistics (Springer-Verlag, New York, 1999), pp. 291–308.
[CrossRef]

Tao, B.

B. Tao, I. Tastl, T. Cooper, M. Blasgen, E. Edwards, “Demosaicing using human visual properties and wavelet interpolation filtering,” in Proceedings of the IS&T/SID 7th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 252–256.

Tastl, I.

B. Tao, I. Tastl, T. Cooper, M. Blasgen, E. Edwards, “Demosaicing using human visual properties and wavelet interpolation filtering,” in Proceedings of the IS&T/SID 7th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 252–256.

J. Holm, I. Tastl, L. Hanlon, P. Hubel, “Color processing for digital photography,” in Colour Engineering: Achieving Device Independent Colour, P. Green, L. MacDonald, eds. (Wiley, New York, 2002), pp. 179–217.

Tietz, J. D.

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Image capture: simulation of sensor responses from hyperspectral images,” IEEE Trans. Image Process. 10, 307–316 (2001).
[CrossRef]

Trussell, H. J.

H. J. Trussell, R. E. Hartwig, “Mathematics for demosaicing,” IEEE Trans. Image Process. 11, 485–492 (2002).
[CrossRef]

Viggiano, J. A. S.

J. A. S. Viggiano, “Minimal-knowledge assumptions in digital still camera characterization. I: Uniform distribution, Toeplitz correlation,” in Proceedings of the IS&T/SID 9th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 2001), pp. 332–336.

Vora, P. L.

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Image capture: simulation of sensor responses from hyperspectral images,” IEEE Trans. Image Process. 10, 307–316 (2001).
[CrossRef]

Vrhel, M. J.

M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).

Wandell, B. A.

F. Xiao, J. M. DiCarlo, P. B. Catrysse, B. A. Wandell, “High dynamic range imaging of natural scenes,” in Final Program and Proceedings of the 10th IS&T/SID Color Imaging Conference: Color Science, Systems and Applications (Society for Imaging Science and Technology, Springfield, Va., 2002), pp. 337–342.

Weiss, Y.

Y. Weiss, E. H. Adelson, “Slow and smooth: a Bayesian theory for the combination of local motion signals in human vision,” (MIT, Cambridge, Mass., 1998).

Xiao, F.

F. Xiao, J. M. DiCarlo, P. B. Catrysse, B. A. Wandell, “High dynamic range imaging of natural scenes,” in Final Program and Proceedings of the 10th IS&T/SID Color Imaging Conference: Color Science, Systems and Applications (Society for Imaging Science and Technology, Springfield, Va., 2002), pp. 337–342.

Yang, D.

D. Yang, B. Fowler, “A 640×512 CMOS image sensor with ultrawide dynamic range floating-point pixel-level ADC,” IEEE J. Solid-State Circuits 34, 1821–1834 (1999).
[CrossRef]

Zhang, X.

X. Zhang, D. H. Brainard, “Bayes color correction method for non-colorimetric digital image sensors,” in Proceedings of the IS&T/SID 12th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 2004).

X. Zhang, D. H. Brainard, “Method and apparatus for estimating true color values for saturated color values in digitally captured image data,” U.S. patent6,731,794 (May4, 2004).

Color Res. Appl. (1)

M. J. Vrhel, R. Gershon, L. S. Iwan, “Measurement and analysis of object reflectance spectra,” Color Res. Appl. 19, 4–9 (1994).

IEEE J. Solid-State Circuits (1)

D. Yang, B. Fowler, “A 640×512 CMOS image sensor with ultrawide dynamic range floating-point pixel-level ADC,” IEEE J. Solid-State Circuits 34, 1821–1834 (1999).
[CrossRef]

IEEE Trans. Consum. Electron. (1)

R. Kakarala, Z. Baharav, “Adaptive demosaicing with the principal vector method,” IEEE Trans. Consum. Electron. 48, 932–937 (2002).
[CrossRef]

IEEE Trans. Image Process. (3)

H. J. Trussell, R. E. Hartwig, “Mathematics for demosaicing,” IEEE Trans. Image Process. 11, 485–492 (2002).
[CrossRef]

R. Kimmel, “Demosaicing: image reconstruction from color CCD samples,” IEEE Trans. Image Process. 8, 1221–1228 (1999).
[CrossRef]

P. L. Vora, J. E. Farrell, J. D. Tietz, D. H. Brainard, “Image capture: simulation of sensor responses from hyperspectral images,” IEEE Trans. Image Process. 10, 307–316 (2001).
[CrossRef]

IEEE Trans. Visualization Comput. Graph. (1)

G. W. Larson, H. Rushmeier, C. Piatko, “A visibility matching tone reproduction operator for high dynamic range scenes,” IEEE Trans. Visualization Comput. Graph. 3, 291–306 (1997).
[CrossRef]

J. Opt. Soc. Am. A (1)

Other (22)

G. D. Finlayson, M. S. Drew, “The maximum ignorance assumption with positivity,” in Proceedings of the IS&T/SID 4th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 202–204.

D. H. Brainard, D. Sherman, “Reconstructing images from trichromatic samples: from basic research to practical applications,” in Proceedings of the 3rd IS&T/SID Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1995), pp. 4–10.

B. Tao, I. Tastl, T. Cooper, M. Blasgen, E. Edwards, “Demosaicing using human visual properties and wavelet interpolation filtering,” in Proceedings of the IS&T/SID 7th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1999), pp. 252–256.

J. A. S. Viggiano, “Minimal-knowledge assumptions in digital still camera characterization. I: Uniform distribution, Toeplitz correlation,” in Proceedings of the IS&T/SID 9th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 2001), pp. 332–336.

D. H. Brainard, Colorimetry (McGraw-Hill, New York, 1995), pp. 26.21–26.54.

J. Holm, “Photographic tone and colour reproduction goals,” in Proceedings of the CIE Expert Symposium on Colour Standards for Image Technology (Bureau Central de la CIE, Vienna, 1996), pp. 51–56.

CIE, “Industrial colour-difference evaluation,” (Bureau Central de la CIE, Vienna, 1995).

S. Pattanaik, J. Ferwerda, M. Fairchild, D. Greenberg, “A multiscale model of adaptation and spatial vision for realistic image display,” in Proceedings of SIGGRAPH’98 ( www.siggraph.org , 1998), pp. 287–298.

F. Xiao, J. M. DiCarlo, P. B. Catrysse, B. A. Wandell, “High dynamic range imaging of natural scenes,” in Final Program and Proceedings of the 10th IS&T/SID Color Imaging Conference: Color Science, Systems and Applications (Society for Imaging Science and Technology, Springfield, Va., 2002), pp. 337–342.

S. K. Nayar, T. Mitsunaga, “High dynamic range imaging: spatially varying pixel exposures,” in Proceedings of IEEE CVPR (IEEE Press, Piscataway, N.J., 2000), pp. 1472–1479.

P. Longere, D. H. Brainard, “Simulation of digital camera images from hyperspectral input,” in Vision Models and Applications to Image and Video Processing, C. van den Branden Lambrecht, ed. (Kluwer Academic, Boston, Mass., 2001), pp. 123–150.

J. Holm, I. Tastl, L. Hanlon, P. Hubel, “Color processing for digital photography,” in Colour Engineering: Achieving Device Independent Colour, P. Green, L. MacDonald, eds. (Wiley, New York, 2002), pp. 179–217.

J. Holm, “A strategy for pictorial digital image processing,” in Proceedings of the IS&T/SID 4th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 1996), pp. 194–201.

E. P. Simoncelli, “Bayesian denoising of visual images in the wavelet domain,” in Bayesian Inference in Wavelet Based Models, P. Müller, B. Vidakovic, eds., Vol. 141 of Lecture Notes in Statistics (Springer-Verlag, New York, 1999), pp. 291–308.
[CrossRef]

Y. Weiss, E. H. Adelson, “Slow and smooth: a Bayesian theory for the combination of local motion signals in human vision,” (MIT, Cambridge, Mass., 1998).

X. Zhang, D. H. Brainard, “Method and apparatus for estimating true color values for saturated color values in digitally captured image data,” U.S. patent6,731,794 (May4, 2004).

T. O. Berger, Statistical Decision Theory and Bayesian Analysis (Springer-Verlag, New York, 1985).

P. M. Lee, Bayesian Statistics (Oxford U. Press, London, 1989).

D. H. Brainard, “Bayesian method for reconstructing color images from trichromatic samples,” in Proceedings of the IS&T 47th Annual Meeting (Society for Imaging Science and Technology, Springfield, Va., 1994), pp. 375–380.

W. R. Dillon, M. Goldstein, Multivariate Analysis (Wiley, New York, 1984).

X. Zhang, D. H. Brainard, “Bayes color correction method for non-colorimetric digital image sensors,” in Proceedings of the IS&T/SID 12th Color Imaging Conference (Society for Imaging Science and Technology, Springfield, Va., 2004).

J. E. Adams, J. F. Hamilton, “Adaptive color plane interpolation in single sensor color electronic camera,” U.S. patent5652621 (July29, 1997).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1
Fig. 1

Kodak DCS200 image rendered (a) without any saturation fix, (b) with pixel-value clipping to a known white point, and (c) with saturation fix for an image with a large number of saturated pixels. Pixels at the face areas of the images are scrambled to protect the privacy of the subjects.

Fig. 2
Fig. 2

Plot of R versus G sensor values for 500 nonsaturated pixels randomly selected from an image (shown at lower right corner) after bilinear demosaicking. This image is a subregion of the truck image in Fig. 1(a). The correlation between the R and the G values for this image area is 0.96. The RG correlation for the full image in Fig. 1(a) is higher (0.99) because the content of that image is largely achromatic.

Fig. 3
Fig. 3

Another DCS200 image rendered (a) without saturation fix, (b) with pixel-value clipping to a known white point, and (c) with saturation fix.

Fig. 4
Fig. 4

Image rendered (a) without saturation fix, (b) with pixel-value clipping to a known white point, and (c) with saturation fix. Saturated pixels were generated by adjusting gains and then clipping in the R and G color channels of an original image without any pixel saturation. This allows for comparison of the true values of the R and G pixels and the estimated values from our saturation-fix algorithm.

Fig. 5
Fig. 5

Comparison of true sensor values versus the saturated-and-fixed sensor values for the R car image in Fig. 4. In the plot, a pixel value of 1 represent the saturation level.

Tables (2)

Tables Icon

Table 1 Color Errors (in ΔE94) for DCS420 Simulation

Tables Icon

Table 2 Color Errors (in ΔE94) for CMOS Sensor Simulation

Equations (16)

Equations on this page are rendered with MathJax. Learn more.

XsXkNμsμk, SsSksSskSk,
XsXk+ek=XsYkNμsμk, SsSksSskSk+Sek,
μxs=μs+Ssk(Sk+Sek)-1(k-μk),
Sxs=Ss-Ssk(Sk+Sek)-1SskT.
P(Xs=x|Yk=k, Yss)=P(Yss|Yk=k, Xs=x)P(Xs=x|Yk=k)P(Yss|Yk=k)=P(Xs+ess|Yk=k, Xs=x)P(Xs=x|Yk=k)P(Yss|Yk=k)=P(ess-x|Yk=k)P(Xs=x|Yk=k)P(Yss|Yk=k)=P(ess-x)P(Xs=x|Yk=k)P(Yss|Yk=k).
μys=μs+Ssk(Sk+Sek)-1(k-μk)=μxs,
Sys=Ss+Ses-Ssk(Sk+Sek)-1SskT=Sxs+Ses.
P(Xs=x|Yk=k, Yss)=P(ess-x|es=0)P(Xs=x|Yk=k)P(Yss|Yk=k)
=P(Xs=x|Yk=k)P(Yss|Yk=k)xs0x<s
=(2π|Sxs|)-1/2exp[-12(x-μxs)TSxs-1(x-μxs)]s(2π|Sxs|)-1/2exp[-12(x-μxs)TSxs-1(x-μxs)]dxxs0x<s
=exp[-12(x-μxs)TSxs-1(x-μxs)]sexp[-12(x-μxs)TSxs-1(x-μxs)]dxxs0x<s
E[Xs|Yk=k, Yss]
=sx exp[-12(x-μxs)TSxs-1(x-μxs)]dxsexp[-12(x-μxs)TSxs-1(x-μxs)]dx.
E(Xs|Yk=k, Yss)
=E(Xs|Yk=k, Xss)=μxs+E(Xs-μxs|Yk=k, Xs-μxss-μxs)=μxs+s-μxsx exp-12Sxs x2dxs-μxsexp-12Sxs x2dx=μxs+-Sxsexp-x22Sxss-μxss-μxsexp-12Sxs x2dx=μxs+Sxsexp-(s-μxs)22Sxss-μxsexp-12Sxs x2dx=μxs+1ZSxs2π1/2exp-(s-μxs)22Sxs,
Z=12πSxss-μxsexp-x22Sxsdx.

Metrics