Abstract

Human observers can distinguish the albedo of real-world surfaces even when the surfaces are viewed in isolation, contrary to the Gelb effect. We sought to measure this ability and to understand the cues that might underlie it. We took photographs of complex surfaces such as stucco and asked observers to judge their diffuse reflectance by comparing them to a physical Munsell scale. Their judgments, while imperfect, were highly correlated with the true reflectance. The judgments were also highly correlated with certain image statistics, such as moment and percentile statistics of the luminance and subband histograms. When we digitally manipulated these statistics in an image, human judgments were correspondingly altered. Moreover, linear combinations of such statistics allow a machine vision system (operating within the constrained world of single surfaces) to estimate albedo with an accuracy similar to that of human observers. Taken together, these results indicate that some simple image statistics have a strong influence on the judgment of surface reflectance.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |

  1. A.Gelb, “Die 'Farbenkonstanz' dez Sehdinge [Color constancy of visual objects],” in Handbuchder normalen und pathologischen Psychologie, W.A.von Bethe, ed. (Springer, 1929), pp. 594-678.
    [CrossRef]
  2. I. Motoyoshi, S. Nishida, L. Sharan, and E. H. Adelson, “Image statistics and the perception of surface qualities,” Nature 447, 206-209 (2007).
    [CrossRef] [PubMed]
  3. A. L. Gilchrist, “The perception of surface blacks and whites,” Sci. Am. 240, 112-124 (1979).
    [CrossRef] [PubMed]
  4. A. L. Gilchrist and A. Jacobsen, “Perception of lightness and illumination in a world of one reflectance,” Perception 13, 5-19 (1984).
    [CrossRef] [PubMed]
  5. E. H. Land and J. J. McCann, “Lightness and retinex theory,” J. Opt. Soc. Am. 61, 1-11 (1971).
    [CrossRef] [PubMed]
  6. A. L. Gilchrist, “Perceived lightness depends on perceived spatial arrangement,” Science 195, 185-187 (1977).
    [CrossRef] [PubMed]
  7. E. H. Adelson, “Perceptual organization and judgment of brightness,” Science 262, 2042-2044 (1993).
    [CrossRef] [PubMed]
  8. M. D'Zmura and G. Iverson, “Color constancy. I. Basic theory of two-stage linear recovery of spectral descriptions for lights and surfaces,” J. Opt. Soc. Am. A 10, 2148-2165 (1993).
    [CrossRef]
  9. D. H. Brainard, “Color constancy in the nearly natural image. 2. Achromatic loci,” J. Opt. Soc. Am. A 15, 307-325 (1998).
    [CrossRef]
  10. A. L. Gilchrist, C. Kossyfidis, F. Bonato, T. Agostini, J. Cataliotti, X. Li, B. Spehar, V. Annan, and E. Economou, “An anchoring theory of lightness perception,” Psychol. Rev. 106, 795-834 (1999).
    [CrossRef] [PubMed]
  11. M. G. Bloj, D. Kersten, and A. C. Hurlbert, “Perception of three-dimensional shape influences color perception through mutual illumination,” Nature 402, 877-879 (1999).
  12. H. Boyaci, L. T. Maloney, and S. Hersh, “The effect of perceived surface orientation on perceived surface albedo in binocularly viewed scenes,” J. Vision 3, 541-553 (2003).
    [CrossRef]
  13. D. H. Brainard, J. M. Kraft, and P. Longere, “Color constancy: developing empirical tests of computational models,” in Color Perception: From Light to Object, R.Mausfeld and D.Heyer, eds. (Oxford U. Press, 2003), pp. 307-334.
  14. L. T. Maloney and J. N. Yang, “The illumination estimation hypothesis and surface color perception,” in Color Vision: Connecting the Mind to the Physical World, R.Mausfeld and D.Heyer, eds. (Oxford U. Press, 2003), pp. 335-358.
  15. S. Nishida and M. Shinya, “Use of image-based information in judgments of surface reflectance properties,” J. Opt. Soc. Am. A 15, 2951-2965 (1998).
    [CrossRef]
  16. R. O. Dror, E. H. Adelson, and A. S. Willsky, “Recognition of surface reflectance properties from a single image under unknown real-world illumination,” in Proceedings of the IEEE Workshop on Identifying Objects across Variation in Lighting: Psychophysics and Computation (IEEE, 2001), available at http://web.mit.edu/persci/people/adelson/pub_pdfs/dror_cvpr01_goem.pdf.
    [PubMed]
  17. R. O. Dror, “Surface reflectance recognition and real-world illumination statistics,” Ph.D. dissertation (Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2002).
  18. R. W. Fleming, R. O. Dror, and E. H. Adelson, “Real-world illumination and the perception of surface reflectance properties,” J. Vision 3, 347-368 (2003).
    [CrossRef]
  19. R. Robilotto and Q. Zaidi, “Limits of lightness identification of real objects under natural viewing conditions,” J. Vision 4, 779-797 (2004).
    [CrossRef]
  20. R. Robilotto and Q. Zaidi, “Lightness identification of patterned three-dimensional, real objects,” J. Vision 6, 18-36 (2006).
    [CrossRef]
  21. J. Beck and S. Prazdny, “Highlights and the perception of glossiness,” Percept. Psychophys. 30, 401-410 (1981).
    [CrossRef]
  22. F. Pellacini, J. A. Ferwerda, and D. P. Greenberg, “Towards a psychophysically-based light reflection model for image synthesis,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, J.R.Brown and K.Akeley, eds. (ACM Press, 2000), pp. 55-64.
  23. J. T. Todd, J. F. Norman, and E. Mingolla, “Lightness constancy in the presence of specular highlights,” Psychol. Sci. 15, 33-39 (2004).
    [CrossRef] [PubMed]
  24. B.-T. Phong, “Illumination for computer generated pictures,” Commun. ACM 18, 311-317 (1975).
    [CrossRef]
  25. Y. Sato, M. D. Wheeler, and K. Ikeuchi, “Object shape and reflectance modeling from observation,” in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, G.O.Owen, T.Whitted, and B.Mones-Hattal, eds. (ACM Press, 1997), pp. 379-387.
  26. Y. Yu and J. Malik, “Recovering photometric properties of architectural scenes from photographs,” in Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, S.Cunningham, W.Bransford, and M.F.Cohen, eds. (ACM Press, 1998), pp. 207-217.
  27. S. R. Marschner, S. H. Westin, E. P. F. Lafortune, K. E. Torrance, and D. P. Green, “Image-based BRDF measurement including human skin,” in Proceedings of the 10th Eurographics Workshop on Rendering, D.Lischinski and G.W.Larson, eds. (Springer, 1999), pp. 139-152.
  28. Y. Yu, P. Debevec, J. Malik, and T. Hawkins, “Inverse global illumination: recovering reflectance models of real scenes from photographs,” in Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, W.Waggenspack, ed. (ACM Press, 1999), pp. 215-224.
  29. S. Boivin and A. Gagalowicz, “Image based rendering of diffuse, specular and glossy surfaces from a single image,” in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, L.Pocock, ed. (ACM Press, 2001), pp. 107-116.
  30. S. Tominaga and N. Tanaka, “Estimating reflection parameters from a single color image,” IEEE Comput. Graphics Appl. 20, 58-66 (2000).
    [CrossRef]
  31. K. Nishino, Z. Zhang, and K. Ikeuchi, “Determining reflectance parameters and illumination distributions from a sparse set of images for view-dependent image synthesis,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2001), pp. 599-601.
  32. R. Ramamoorthi and P. Hanrahan, “A signal processing framework for inverse rendering,” in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, L.Pocock, ed. (ACM Press, 2001), pp. 117-128.
  33. P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, “Acquiring the reflectance field of a human face,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, J.R.Brown and K.Akeley, eds. (ACM Press, 2000), pp. 145-156.
  34. P. Debevec, C. Tchou, A. Gardner, T. Hawkins, C. Poullis, J. Stumpfel, A. Jones, N. Yun, P. Einarsson, T. Lundgren, M. Fajardo, and P. Martinez, “Estimating surface reflectance properties of a complex scene under captured natural illumination,” Tech. Rep. ICT-TR-06 (University of Southern California Institute for Creative Technologies Graphics Laboratory, 2004).
  35. J. J. Koenderink, A. J. Van Doorn, K. J. Dana, and S. Nayar, “Bidirectional reflectance distribution function of thoroughly pitted surfaces,” Int. J. Comput. Vis. 31, 129-144 (1999).
    [CrossRef]
  36. J. J. Koenderink and A. J. van Doorn, “Illuminance texture due to surface mesostructure,” J. Opt. Soc. Am. A 13, 452-463 (1996).
    [CrossRef]
  37. D. Coffin, “Raw digital photo decoding in Linux,” http://www.cybercom.net/~dcoffin/dcraw.
  38. P. Debevec, C. Tchou, and T. Hawkins, “HDRShop: high dynamic range image processing and manipulation,” http://www.hdrshop.com.
  39. P. Burt and E. H. Adelson, “Laplacian pyramid as a compact image code,” IEEE Trans. Commun. 31, 532-540 (1983).
    [CrossRef]
  40. E. P. Simoncelli and W. T. Freeman, “The steerable pyramid: a flexible architecture for multi-scale derivative computation,” in Proceedings of IEEE Conference on Image Processing (IEEE, 1995), pp. 444-447.
    [CrossRef]
  41. C.-C. Chang and C.-J. Lin, “LIBSVM: a library for support vector machines,” http://www.csie.ntu.edu.tw/~cjlin/libsvm.
  42. Columbia-Utrecht Reflectance and Texture (CURET) database, http://www.cs.columbia.edu/CAVE/curet.
  43. D. J. Heeger and J. R. Bergen, “Pyramid-based texture analysis/synthesis,” in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, R.Cook, ed. (ACM Press, 1995), pp. 229-238.
  44. M. D. Rutherford and D. H. Brainard, “Lightness constancy: a direct test of the illumination estimation hypothesis,” Psychol. Sci. 13, 142-149 (2002).
    [CrossRef] [PubMed]
  45. C. Chubb, M. S. Landy, and J. Econopouly, “A visual mechanism tuned to black,” Vision Res. 44, 3223-3232 (2004).
    [CrossRef] [PubMed]
  46. B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature 381, 607-609 (1996).
    [CrossRef] [PubMed]
  47. E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci. 24, 1193-1216 (2001).
    [CrossRef] [PubMed]
  48. M. Varma and A. Zisserman, “A statistical approach to texture classification from single images,” Int. J. Comput. Vis. 62, 61-81 (2005).
  49. M. Oren and S. K. Nayar, “Generalization of the Lambertian model and implications for machine vision,” Int. J. Comput. Vis. 14, 227-251 (1995).
    [CrossRef]

2007

I. Motoyoshi, S. Nishida, L. Sharan, and E. H. Adelson, “Image statistics and the perception of surface qualities,” Nature 447, 206-209 (2007).
[CrossRef] [PubMed]

2006

R. Robilotto and Q. Zaidi, “Lightness identification of patterned three-dimensional, real objects,” J. Vision 6, 18-36 (2006).
[CrossRef]

2005

M. Varma and A. Zisserman, “A statistical approach to texture classification from single images,” Int. J. Comput. Vis. 62, 61-81 (2005).

2004

J. T. Todd, J. F. Norman, and E. Mingolla, “Lightness constancy in the presence of specular highlights,” Psychol. Sci. 15, 33-39 (2004).
[CrossRef] [PubMed]

C. Chubb, M. S. Landy, and J. Econopouly, “A visual mechanism tuned to black,” Vision Res. 44, 3223-3232 (2004).
[CrossRef] [PubMed]

R. Robilotto and Q. Zaidi, “Limits of lightness identification of real objects under natural viewing conditions,” J. Vision 4, 779-797 (2004).
[CrossRef]

2003

R. W. Fleming, R. O. Dror, and E. H. Adelson, “Real-world illumination and the perception of surface reflectance properties,” J. Vision 3, 347-368 (2003).
[CrossRef]

H. Boyaci, L. T. Maloney, and S. Hersh, “The effect of perceived surface orientation on perceived surface albedo in binocularly viewed scenes,” J. Vision 3, 541-553 (2003).
[CrossRef]

2002

M. D. Rutherford and D. H. Brainard, “Lightness constancy: a direct test of the illumination estimation hypothesis,” Psychol. Sci. 13, 142-149 (2002).
[CrossRef] [PubMed]

2001

E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci. 24, 1193-1216 (2001).
[CrossRef] [PubMed]

2000

S. Tominaga and N. Tanaka, “Estimating reflection parameters from a single color image,” IEEE Comput. Graphics Appl. 20, 58-66 (2000).
[CrossRef]

1999

J. J. Koenderink, A. J. Van Doorn, K. J. Dana, and S. Nayar, “Bidirectional reflectance distribution function of thoroughly pitted surfaces,” Int. J. Comput. Vis. 31, 129-144 (1999).
[CrossRef]

A. L. Gilchrist, C. Kossyfidis, F. Bonato, T. Agostini, J. Cataliotti, X. Li, B. Spehar, V. Annan, and E. Economou, “An anchoring theory of lightness perception,” Psychol. Rev. 106, 795-834 (1999).
[CrossRef] [PubMed]

M. G. Bloj, D. Kersten, and A. C. Hurlbert, “Perception of three-dimensional shape influences color perception through mutual illumination,” Nature 402, 877-879 (1999).

1998

1996

J. J. Koenderink and A. J. van Doorn, “Illuminance texture due to surface mesostructure,” J. Opt. Soc. Am. A 13, 452-463 (1996).
[CrossRef]

B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature 381, 607-609 (1996).
[CrossRef] [PubMed]

1995

M. Oren and S. K. Nayar, “Generalization of the Lambertian model and implications for machine vision,” Int. J. Comput. Vis. 14, 227-251 (1995).
[CrossRef]

1993

1984

A. L. Gilchrist and A. Jacobsen, “Perception of lightness and illumination in a world of one reflectance,” Perception 13, 5-19 (1984).
[CrossRef] [PubMed]

1983

P. Burt and E. H. Adelson, “Laplacian pyramid as a compact image code,” IEEE Trans. Commun. 31, 532-540 (1983).
[CrossRef]

1981

J. Beck and S. Prazdny, “Highlights and the perception of glossiness,” Percept. Psychophys. 30, 401-410 (1981).
[CrossRef]

1979

A. L. Gilchrist, “The perception of surface blacks and whites,” Sci. Am. 240, 112-124 (1979).
[CrossRef] [PubMed]

1977

A. L. Gilchrist, “Perceived lightness depends on perceived spatial arrangement,” Science 195, 185-187 (1977).
[CrossRef] [PubMed]

1975

B.-T. Phong, “Illumination for computer generated pictures,” Commun. ACM 18, 311-317 (1975).
[CrossRef]

1971

Annu. Rev. Neurosci.

E. P. Simoncelli and B. A. Olshausen, “Natural image statistics and neural representation,” Annu. Rev. Neurosci. 24, 1193-1216 (2001).
[CrossRef] [PubMed]

Commun. ACM

B.-T. Phong, “Illumination for computer generated pictures,” Commun. ACM 18, 311-317 (1975).
[CrossRef]

IEEE Comput. Graphics Appl.

S. Tominaga and N. Tanaka, “Estimating reflection parameters from a single color image,” IEEE Comput. Graphics Appl. 20, 58-66 (2000).
[CrossRef]

IEEE Trans. Commun.

P. Burt and E. H. Adelson, “Laplacian pyramid as a compact image code,” IEEE Trans. Commun. 31, 532-540 (1983).
[CrossRef]

Int. J. Comput. Vis.

J. J. Koenderink, A. J. Van Doorn, K. J. Dana, and S. Nayar, “Bidirectional reflectance distribution function of thoroughly pitted surfaces,” Int. J. Comput. Vis. 31, 129-144 (1999).
[CrossRef]

M. Varma and A. Zisserman, “A statistical approach to texture classification from single images,” Int. J. Comput. Vis. 62, 61-81 (2005).

M. Oren and S. K. Nayar, “Generalization of the Lambertian model and implications for machine vision,” Int. J. Comput. Vis. 14, 227-251 (1995).
[CrossRef]

J. Opt. Soc. Am.

J. Opt. Soc. Am. A

J. Vision

R. W. Fleming, R. O. Dror, and E. H. Adelson, “Real-world illumination and the perception of surface reflectance properties,” J. Vision 3, 347-368 (2003).
[CrossRef]

R. Robilotto and Q. Zaidi, “Limits of lightness identification of real objects under natural viewing conditions,” J. Vision 4, 779-797 (2004).
[CrossRef]

R. Robilotto and Q. Zaidi, “Lightness identification of patterned three-dimensional, real objects,” J. Vision 6, 18-36 (2006).
[CrossRef]

H. Boyaci, L. T. Maloney, and S. Hersh, “The effect of perceived surface orientation on perceived surface albedo in binocularly viewed scenes,” J. Vision 3, 541-553 (2003).
[CrossRef]

Nature

I. Motoyoshi, S. Nishida, L. Sharan, and E. H. Adelson, “Image statistics and the perception of surface qualities,” Nature 447, 206-209 (2007).
[CrossRef] [PubMed]

M. G. Bloj, D. Kersten, and A. C. Hurlbert, “Perception of three-dimensional shape influences color perception through mutual illumination,” Nature 402, 877-879 (1999).

B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature 381, 607-609 (1996).
[CrossRef] [PubMed]

Percept. Psychophys.

J. Beck and S. Prazdny, “Highlights and the perception of glossiness,” Percept. Psychophys. 30, 401-410 (1981).
[CrossRef]

Perception

A. L. Gilchrist and A. Jacobsen, “Perception of lightness and illumination in a world of one reflectance,” Perception 13, 5-19 (1984).
[CrossRef] [PubMed]

Psychol. Rev.

A. L. Gilchrist, C. Kossyfidis, F. Bonato, T. Agostini, J. Cataliotti, X. Li, B. Spehar, V. Annan, and E. Economou, “An anchoring theory of lightness perception,” Psychol. Rev. 106, 795-834 (1999).
[CrossRef] [PubMed]

Psychol. Sci.

J. T. Todd, J. F. Norman, and E. Mingolla, “Lightness constancy in the presence of specular highlights,” Psychol. Sci. 15, 33-39 (2004).
[CrossRef] [PubMed]

M. D. Rutherford and D. H. Brainard, “Lightness constancy: a direct test of the illumination estimation hypothesis,” Psychol. Sci. 13, 142-149 (2002).
[CrossRef] [PubMed]

Sci. Am.

A. L. Gilchrist, “The perception of surface blacks and whites,” Sci. Am. 240, 112-124 (1979).
[CrossRef] [PubMed]

Science

A. L. Gilchrist, “Perceived lightness depends on perceived spatial arrangement,” Science 195, 185-187 (1977).
[CrossRef] [PubMed]

E. H. Adelson, “Perceptual organization and judgment of brightness,” Science 262, 2042-2044 (1993).
[CrossRef] [PubMed]

Vision Res.

C. Chubb, M. S. Landy, and J. Econopouly, “A visual mechanism tuned to black,” Vision Res. 44, 3223-3232 (2004).
[CrossRef] [PubMed]

Other

E. P. Simoncelli and W. T. Freeman, “The steerable pyramid: a flexible architecture for multi-scale derivative computation,” in Proceedings of IEEE Conference on Image Processing (IEEE, 1995), pp. 444-447.
[CrossRef]

C.-C. Chang and C.-J. Lin, “LIBSVM: a library for support vector machines,” http://www.csie.ntu.edu.tw/~cjlin/libsvm.

Columbia-Utrecht Reflectance and Texture (CURET) database, http://www.cs.columbia.edu/CAVE/curet.

D. J. Heeger and J. R. Bergen, “Pyramid-based texture analysis/synthesis,” in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, R.Cook, ed. (ACM Press, 1995), pp. 229-238.

D. H. Brainard, J. M. Kraft, and P. Longere, “Color constancy: developing empirical tests of computational models,” in Color Perception: From Light to Object, R.Mausfeld and D.Heyer, eds. (Oxford U. Press, 2003), pp. 307-334.

L. T. Maloney and J. N. Yang, “The illumination estimation hypothesis and surface color perception,” in Color Vision: Connecting the Mind to the Physical World, R.Mausfeld and D.Heyer, eds. (Oxford U. Press, 2003), pp. 335-358.

D. Coffin, “Raw digital photo decoding in Linux,” http://www.cybercom.net/~dcoffin/dcraw.

P. Debevec, C. Tchou, and T. Hawkins, “HDRShop: high dynamic range image processing and manipulation,” http://www.hdrshop.com.

F. Pellacini, J. A. Ferwerda, and D. P. Greenberg, “Towards a psychophysically-based light reflection model for image synthesis,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, J.R.Brown and K.Akeley, eds. (ACM Press, 2000), pp. 55-64.

Y. Sato, M. D. Wheeler, and K. Ikeuchi, “Object shape and reflectance modeling from observation,” in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, G.O.Owen, T.Whitted, and B.Mones-Hattal, eds. (ACM Press, 1997), pp. 379-387.

Y. Yu and J. Malik, “Recovering photometric properties of architectural scenes from photographs,” in Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, S.Cunningham, W.Bransford, and M.F.Cohen, eds. (ACM Press, 1998), pp. 207-217.

S. R. Marschner, S. H. Westin, E. P. F. Lafortune, K. E. Torrance, and D. P. Green, “Image-based BRDF measurement including human skin,” in Proceedings of the 10th Eurographics Workshop on Rendering, D.Lischinski and G.W.Larson, eds. (Springer, 1999), pp. 139-152.

Y. Yu, P. Debevec, J. Malik, and T. Hawkins, “Inverse global illumination: recovering reflectance models of real scenes from photographs,” in Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, W.Waggenspack, ed. (ACM Press, 1999), pp. 215-224.

S. Boivin and A. Gagalowicz, “Image based rendering of diffuse, specular and glossy surfaces from a single image,” in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, L.Pocock, ed. (ACM Press, 2001), pp. 107-116.

A.Gelb, “Die 'Farbenkonstanz' dez Sehdinge [Color constancy of visual objects],” in Handbuchder normalen und pathologischen Psychologie, W.A.von Bethe, ed. (Springer, 1929), pp. 594-678.
[CrossRef]

R. O. Dror, E. H. Adelson, and A. S. Willsky, “Recognition of surface reflectance properties from a single image under unknown real-world illumination,” in Proceedings of the IEEE Workshop on Identifying Objects across Variation in Lighting: Psychophysics and Computation (IEEE, 2001), available at http://web.mit.edu/persci/people/adelson/pub_pdfs/dror_cvpr01_goem.pdf.
[PubMed]

R. O. Dror, “Surface reflectance recognition and real-world illumination statistics,” Ph.D. dissertation (Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2002).

K. Nishino, Z. Zhang, and K. Ikeuchi, “Determining reflectance parameters and illumination distributions from a sparse set of images for view-dependent image synthesis,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2001), pp. 599-601.

R. Ramamoorthi and P. Hanrahan, “A signal processing framework for inverse rendering,” in Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, L.Pocock, ed. (ACM Press, 2001), pp. 117-128.

P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, “Acquiring the reflectance field of a human face,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, J.R.Brown and K.Akeley, eds. (ACM Press, 2000), pp. 145-156.

P. Debevec, C. Tchou, A. Gardner, T. Hawkins, C. Poullis, J. Stumpfel, A. Jones, N. Yun, P. Einarsson, T. Lundgren, M. Fajardo, and P. Martinez, “Estimating surface reflectance properties of a complex scene under captured natural illumination,” Tech. Rep. ICT-TR-06 (University of Southern California Institute for Creative Technologies Graphics Laboratory, 2004).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (26)

Fig. 1
Fig. 1

(a) In the Gelb demonstration a smooth Lambertian black surface can look the same as a smooth Lambertian white surface. (b) Gelb effect fails for complex surfaces. The stucco samples have the same mean luminance, yet it is easy to tell the white stucco from the black.

Fig. 2
Fig. 2

Stimuli used for studying reflectance perception: (a) Wallach’s disc annulus displays, (b) Mondrian-like displays with flat Lambertian surfaces, (c) Fleming et al.’s simulated spheres in complex real-world illumination [18], (d) simulated locally smooth bumpy surfaces used by Nishida and Shinya [15].

Fig. 3
Fig. 3

Examples of surfaces in our data set. All surfaces shown here were photographed under an overhead fluorescent light.

Fig. 4
Fig. 4

Image data acquisition. Three indoor lighting conditions were used: (a) light 1, overhead fluorescent light source; (b) light 2, focused halogen spotlight; (c) light 3, diffuse tungsten halogen lamp. (d) Schematic layout of the setup. Two views, one from the side and one from front, are shown. (e), (f) Red and blue color components of an orange surface look like white and black surfaces, respectively. (g) Ground truth is acquired using a uniformly illuminated flat material sample and a standard white surface. A user clicks on two regions, one on the sample and one on the standard. The ratio of mean luminance of the two regions is used to calculate the albedo for each color channel.

Fig. 5
Fig. 5

Luminance histograms of light and dark materials exhibit systematic differences. (a) Light modeling clay; (d) dark stucco; and (b), (c) the respective luminance histograms. (c), (f) Standard deviation and skewness of the log-luminance histogram is plotted against the ground truth for albedo for all the surfaces in our data set. All plots pertain to the overhead fluorescent lighting condition.

Fig. 6
Fig. 6

(a) ROC curves for the 90th percentile, standard deviation, and skewness of log-luminance values. These statistics perform well above chance at the task of classifying surfaces as light or dark. (b) Skewness of the log-luminance histogram is plotted against the physical diffuse reflectance. A linear regression model is a poor fit ( p < 0.05 , r 2 = 0.27 ). (c) Applying a log transformation to both axes of (b) improves the fit of the linear regression model ( p < 0.05 , r 2 = 0.34 ). (c) Log of 90th percentile of the log-luminance histogram is plotted against the log of the diffuse reflectance. The linear fit is still not very good ( p < 0.05 , r 2 = 0.29 ). Statistics were pooled across all lighting conditions for all plots in this figure.

Fig. 7
Fig. 7

Pixel histograms of filtered images look different for white and black surfaces. (a) Light modeling clay from Fig. 5; (b) output of Laplacian of Gaussian filter ( σ = 0.5 of size 5 × 5 pixels); (c) output of horizontal Sobel filter ( 3 × 3 pixels); (d) dark stucco from Fig. 5. (e), (f) LoG and Sobel filter outputs; (g) pixel histograms of images in (b) and (e); (h) pixel histograms of images in (c) and (f). All plots pertain to the overhead fluorescent lighting condition.

Fig. 8
Fig. 8

ROC curves for statistics of filter outputs: (a) 90th percentile, skewness, and standard deviation of LoG filter output; (b) 90th percentile and standard deviation for Sobel filter output are significantly above chance; (c), (d) plot the log of the 90th percentile of the LoG and Sobel filter outputs, respectively, against the log of physical diffuse reflectance. The linear regression fits are shown as black lines ( p < 0.05 ) ; r 2 values are 0.63 and 0.62 for (c) and (d), respectively.

Fig. 9
Fig. 9

(a) Standard deviation of log-luminance and the 90th percentile of LoG filter output are correlated ( r = 0.6237 , p < 0.05 ). (b), (c), (d) Outputs of three linear models are plotted against the true diffuse reflectance for a subset of our surfaces. Model A uses one statistic, the standard deviation of LoG filter outputs to predict the albedo of a surface. Model B uses two statistics—standard deviation and 10th percentile of LoG filter outputs. Model C uses three statistics—standard deviation and the 10th and 90th percentiles of LoG filter outputs. For all three cases, the model ratings were averaged over all three lighting conditions. The error bars indicate the minimum and maximum ratings. If the models were perfect at predicting physical albedo, all points would lie along the black line with slope=1. The slopes of the best fit lines are indicated in each plot. The asterisk denotes statistical significance ( p < 0.05 ) . The r 2 statistic is similar for all plots—0.42 for (b), 0.38 for (c), and 0.40 for (d).

Fig. 10
Fig. 10

Output of the regression technique is plotted versus ground truth for diffuse reflectance (albedo). Bars indicate maximum and minimum ratings. If the technique were perfect, all points would lie along the diagonal.

Fig. 11
Fig. 11

Observers viewed stimuli on the LCD panel and matched reflectance properties of the surface on the screen to one of the standard surfaces on the Munsell chart. (a) Dimensions of experimental layout, (b) photograph of the reference box, (c) dimensions of the box.

Fig. 12
Fig. 12

Results of experiment I. Perceived albedo versus ground truth for four observers. Responses were pooled across all lighting conditions. Error bars indicate 95% confidence intervals. The responses of a veridical observer would lie along the black line with slope=1. The gray line is the linear regression fit to each observer’s data. The slope of the best fit line is indicated in each plot. The asterisk denotes significance (p-value < 0.05 ).

Fig. 13
Fig. 13

Observers tend to agree with one another. Perceived reflectance ratings for every pair of observers from Fig. 12 are plotted here. If all observers behaved in the same way, all data points would lie on the black lines with slope=1; r 2 values indicate that there is a great deal of agreement among observers.

Fig. 14
Fig. 14

Surfaces in (a) and (b) have similar diffuse reflectance values (0.085 and 0.092), respectively but dissimilar specular components and surface shapes. (c), (d) Errors in human judgments for the two surfaces seem to vary with the change in specular reflectance and mesostructure. Error bars indicate maximum and minimum errors. The data and photographs in this figure pertain to the overhead fluorescent lighting condition.

Fig. 15
Fig. 15

Effect of lighting conditions. Light 1 is the overhead fluorescent light, light 2 is the halogen spotlight, and light 3 is the diffuse halogen source. (a)–(d) show the averaged observer ratings for four surfaces. Error bars indicate 95% confidence intervals.

Fig. 16
Fig. 16

Observers do not rate individual color channels in the same way. (a) Red channel, (b) green channel, and (c) B channel of an orange surface and the respective averaged observer ratings. (d) Even with the same mean image luminance, illumination conditions, and surface geometry, observers can extract useful information from the image to discern diffuse reflectance.

Fig. 17
Fig. 17

Agreement between observers and the regression technique (the model) is fairly high. The r 2 values here are somewhat lower than the agreement between observers but are not very different (Fig. 13).

Fig. 18
Fig. 18

(a) Light surface and (b) dark surface. (c) Result of matching histogram statistics of (a) to those of (b) and (d) and vice versa.

Fig. 19
Fig. 19

Observer data for histogram-manipulated images (a) through (d) show four different surfaces in one group. The channels R and B refer to the red and blue color channels, respectively, of the color photographs of each surface. The R2B channel is the result of matching the histogram statistics of the R channel image to that of the B channel. B2R is the result of matching the histogram statistics of the B channel image to that of the R channel. The data are pooled across all observers in the group. All plots here are from the overhead fluorescent lighting condition. Error bars are 95% confidence intervals. We note that observers consistently rate R2B similar to B rather than to R and vice versa.

Fig. 20
Fig. 20

Block diagram of the activity-map-based Heeger–Bergen technique.

Fig. 21
Fig. 21

Comparison of histogram-matching techniques: (a) source image, (b) target image, (c) luminance histogram matching, (d) Heeger–Bergen output, (e) activity-map-based Heeger–Bergen output.

Fig. 22
Fig. 22

(a) Some of the CURET images we used. The three rows correspond to materials 11 (plaster), 18 (rug), and 21 (sponge). Three different views are shown for each material. The images were multiplicatively normalized to have the same mean luminance. (b) One of the materials from our data set. The two rows correspond to two viewpoints and the columns to the three different illumination directions. All images were normalized to have the same mean.

Fig. 23
Fig. 23

Effect of varying illumination and viewpoint on image statistics. The skewness of center–surround filtered images is plotted against the ground truth for diffuse reflectance. A Laplacian of a Gaussian filter was used ( σ = 0.5 , size 5 × 5 pixels). (a) Skewness statistics is plotted for all 159 images of the 9 CURET materials against the Oren–Nayar model parameter “rho” [49]. Note the vertical smear at each value of rho. (b) All six images of the nine materials in our data set were used. The x axis is the ground truth for diffuse reflectance for the materials.

Fig. 24
Fig. 24

Perceived reflectance for an observer is plotted against the ground truth. For each material, observer ratings were pooled across all viewing and illumination directions. Error bars are 95% confidence intervals. (a) CURET images: There is no linear relationship between perceived and true reflectance ( p > 0.05 ) . (b) Our images: Observer data can be explained by a linear model ( p < 0.05 , r 2 = 0.85 ). Similar trends were obtained for other observers.

Fig. 25
Fig. 25

Perceived reflectance of an image of a surface is plotted against the skewness of the center–surround filter output. A Laplacian of a Gaussian filter was used ( σ = 0.5 , size 5 × 5 pixels). (a) CURET images and (b) our images. A linear relationship can be observed in both plots. For (c), p < 0.05 , r 2 = 0.42 and for (d), p < 0.05 , r 2 = 0.64 .

Fig. 26
Fig. 26

Verifying linearity correction. Images A and B are the linear outputs of the dcraw program for two exposures of the same scene [37]. The original exposures were recorded in the Canon RAW format (CRW). The exposure time for B was twice that of A. All other camera parameters were the same for A and B. The pixel values of the linearized images are plotted against each other. If the combination of internal camera processing followed by dcraw was perfectly linear, all data points would lie along a line with slope=2. The actual fit is quite good: slope= 2.1055 ± 0.0084 ( p < 0.05 , r 2 = 0.94 ).

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

G = A m o d i f i e d A o r i g ,

Metrics