Abstract

In the classical shape-from-shading model, surface luminance depends primarily on the unit surface normal. However, under diffuse lighting conditions, such as the sky on a cloudy day, luminance depends primarily on the amount of sky that is visible from each surface element, with surface normal of secondary importance. This claim is formalized in terms of a dominating sky principle and a surface aperture function. An approximately functional constraint between surface luminance and aperture emerges. It is shown how to use this constraint to recover a depth map from an image efficiently. A curious difference from the classical shape-from-shading problem is uncovered. When one assumes a point light source, the local geometric constraints of the shape-from-shading problem lie along the surface. However, in the diffuse-lighting problem, the local geometric constraints are found in a visibility field, which is defined in the free space above the surface.

© 1994 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. K. P. Horn, M. J. Brooks, eds., Shape From Shading (MIT, Cambridge, Mass., 1989).
  2. B. K. P. Horn, R. W. Sjoberg, “Calculating the reflectance map,” Appl. Opt. 18, 1770–1779 (1979).
    [Crossref] [PubMed]
  3. E. M. Sparrow, R. D. Cess, Radiation Heat Transfer (McGraw-Hill, New York, 1978).
  4. J. J. Koenderink, A. J. van Doorn, “Geometrical modes as a general method to treat diffuse interreflections in radiometry,”J. Opt. Soc. Am. 73, 843–850 (1983).
    [Crossref]
  5. A. Gershun, “The light field,”J. Math. Phys. 18, 51–151 (1939).
  6. M. V. Klein, T. E. Furtak, Optics (Wiley, New York, 1986).
  7. J. J. Koenderink, W. A. Richards, “Why is snow so bright?” J. Opt. Soc. Am. A 9, 643–648 (1992).
    [Crossref]
  8. C. M. Goral, K. E. Torrence, D. P. Greenberg, B. Battaile, “Modelling the interaction of light between diffuse surfaces,” Comput. Graphics 18, 213–222 (1984).
    [Crossref]
  9. S. K. Nayar, K. Ickeuchi, T. Kanade, “Shape from interreflections,” Int. J. Comput. Vision 6, 173–195 (1991).
    [Crossref]
  10. E. L. Krinov, Spectral Reflectance Properties of Natural Materials, NRC Technical Translation 439 (National Research Council of Canada, Ottawa, 1971).
  11. W. A. Richards, “Lightness scale from intensity distributions,” Appl. Opt. 21, 2569–2582 (1982).
    [Crossref] [PubMed]
  12. J. P. Hailman, “Environmental light and conspicuous colors,” in The Bahavioral Significance of Color, E. H. Burtt, ed. (Garland, New York, 1979).
  13. R. Wehner, “Spatial vision in arthropods,” in Handbook of Sensory Physiology, H. Autrum, ed., Vol. C of Comparative Physiology and Evolution of Vision in Invertebrates (Springer-Verlag, Berlin, 1981).
    [Crossref]
  14. I. M. S. Langer, S. W. Zucker, “Qualitative shape from active shading,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, New York, 1992), pp. 713–715.
  15. B. K. P. Horn, “Understanding image intensities,” Artif. Intell. 8, 201–231 (1977).
    [Crossref]
  16. D. Forsyth, A. Zisserman, “Reflections on shading,”IEEE Trans. Pattern Anal. Mach. Intell. 13, 671–679 (1991).
    [Crossref]
  17. I. M. S. Langer, “The computational geometry of light,” Ph.D. dissertation (McGill University, Montreal, Quebec, Canada, 1994).
  18. E. MacCurdy, ed., The Notebooks of Leonardo da Vinci (Jonathan Cape, London, 1938), p. 332.
  19. K. Nicolaides, The Natural Way to Draw (Houghton Mifflin, Boston, Mass., 1941).

1992 (1)

1991 (2)

S. K. Nayar, K. Ickeuchi, T. Kanade, “Shape from interreflections,” Int. J. Comput. Vision 6, 173–195 (1991).
[Crossref]

D. Forsyth, A. Zisserman, “Reflections on shading,”IEEE Trans. Pattern Anal. Mach. Intell. 13, 671–679 (1991).
[Crossref]

1984 (1)

C. M. Goral, K. E. Torrence, D. P. Greenberg, B. Battaile, “Modelling the interaction of light between diffuse surfaces,” Comput. Graphics 18, 213–222 (1984).
[Crossref]

1983 (1)

1982 (1)

1979 (1)

1977 (1)

B. K. P. Horn, “Understanding image intensities,” Artif. Intell. 8, 201–231 (1977).
[Crossref]

1939 (1)

A. Gershun, “The light field,”J. Math. Phys. 18, 51–151 (1939).

Battaile, B.

C. M. Goral, K. E. Torrence, D. P. Greenberg, B. Battaile, “Modelling the interaction of light between diffuse surfaces,” Comput. Graphics 18, 213–222 (1984).
[Crossref]

Cess, R. D.

E. M. Sparrow, R. D. Cess, Radiation Heat Transfer (McGraw-Hill, New York, 1978).

Forsyth, D.

D. Forsyth, A. Zisserman, “Reflections on shading,”IEEE Trans. Pattern Anal. Mach. Intell. 13, 671–679 (1991).
[Crossref]

Furtak, T. E.

M. V. Klein, T. E. Furtak, Optics (Wiley, New York, 1986).

Gershun, A.

A. Gershun, “The light field,”J. Math. Phys. 18, 51–151 (1939).

Goral, C. M.

C. M. Goral, K. E. Torrence, D. P. Greenberg, B. Battaile, “Modelling the interaction of light between diffuse surfaces,” Comput. Graphics 18, 213–222 (1984).
[Crossref]

Greenberg, D. P.

C. M. Goral, K. E. Torrence, D. P. Greenberg, B. Battaile, “Modelling the interaction of light between diffuse surfaces,” Comput. Graphics 18, 213–222 (1984).
[Crossref]

Hailman, J. P.

J. P. Hailman, “Environmental light and conspicuous colors,” in The Bahavioral Significance of Color, E. H. Burtt, ed. (Garland, New York, 1979).

Horn, B. K. P.

B. K. P. Horn, R. W. Sjoberg, “Calculating the reflectance map,” Appl. Opt. 18, 1770–1779 (1979).
[Crossref] [PubMed]

B. K. P. Horn, “Understanding image intensities,” Artif. Intell. 8, 201–231 (1977).
[Crossref]

Ickeuchi, K.

S. K. Nayar, K. Ickeuchi, T. Kanade, “Shape from interreflections,” Int. J. Comput. Vision 6, 173–195 (1991).
[Crossref]

Kanade, T.

S. K. Nayar, K. Ickeuchi, T. Kanade, “Shape from interreflections,” Int. J. Comput. Vision 6, 173–195 (1991).
[Crossref]

Klein, M. V.

M. V. Klein, T. E. Furtak, Optics (Wiley, New York, 1986).

Koenderink, J. J.

Krinov, E. L.

E. L. Krinov, Spectral Reflectance Properties of Natural Materials, NRC Technical Translation 439 (National Research Council of Canada, Ottawa, 1971).

Langer, I. M. S.

I. M. S. Langer, S. W. Zucker, “Qualitative shape from active shading,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, New York, 1992), pp. 713–715.

I. M. S. Langer, “The computational geometry of light,” Ph.D. dissertation (McGill University, Montreal, Quebec, Canada, 1994).

Nayar, S. K.

S. K. Nayar, K. Ickeuchi, T. Kanade, “Shape from interreflections,” Int. J. Comput. Vision 6, 173–195 (1991).
[Crossref]

Nicolaides, K.

K. Nicolaides, The Natural Way to Draw (Houghton Mifflin, Boston, Mass., 1941).

Richards, W. A.

Sjoberg, R. W.

Sparrow, E. M.

E. M. Sparrow, R. D. Cess, Radiation Heat Transfer (McGraw-Hill, New York, 1978).

Torrence, K. E.

C. M. Goral, K. E. Torrence, D. P. Greenberg, B. Battaile, “Modelling the interaction of light between diffuse surfaces,” Comput. Graphics 18, 213–222 (1984).
[Crossref]

van Doorn, A. J.

Wehner, R.

R. Wehner, “Spatial vision in arthropods,” in Handbook of Sensory Physiology, H. Autrum, ed., Vol. C of Comparative Physiology and Evolution of Vision in Invertebrates (Springer-Verlag, Berlin, 1981).
[Crossref]

Zisserman, A.

D. Forsyth, A. Zisserman, “Reflections on shading,”IEEE Trans. Pattern Anal. Mach. Intell. 13, 671–679 (1991).
[Crossref]

Zucker, S. W.

I. M. S. Langer, S. W. Zucker, “Qualitative shape from active shading,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, New York, 1992), pp. 713–715.

Appl. Opt. (2)

Artif. Intell. (1)

B. K. P. Horn, “Understanding image intensities,” Artif. Intell. 8, 201–231 (1977).
[Crossref]

Comput. Graphics (1)

C. M. Goral, K. E. Torrence, D. P. Greenberg, B. Battaile, “Modelling the interaction of light between diffuse surfaces,” Comput. Graphics 18, 213–222 (1984).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

D. Forsyth, A. Zisserman, “Reflections on shading,”IEEE Trans. Pattern Anal. Mach. Intell. 13, 671–679 (1991).
[Crossref]

Int. J. Comput. Vision (1)

S. K. Nayar, K. Ickeuchi, T. Kanade, “Shape from interreflections,” Int. J. Comput. Vision 6, 173–195 (1991).
[Crossref]

J. Math. Phys. (1)

A. Gershun, “The light field,”J. Math. Phys. 18, 51–151 (1939).

J. Opt. Soc. Am. (1)

J. Opt. Soc. Am. A (1)

Other (10)

B. K. P. Horn, M. J. Brooks, eds., Shape From Shading (MIT, Cambridge, Mass., 1989).

E. M. Sparrow, R. D. Cess, Radiation Heat Transfer (McGraw-Hill, New York, 1978).

M. V. Klein, T. E. Furtak, Optics (Wiley, New York, 1986).

E. L. Krinov, Spectral Reflectance Properties of Natural Materials, NRC Technical Translation 439 (National Research Council of Canada, Ottawa, 1971).

I. M. S. Langer, “The computational geometry of light,” Ph.D. dissertation (McGill University, Montreal, Quebec, Canada, 1994).

E. MacCurdy, ed., The Notebooks of Leonardo da Vinci (Jonathan Cape, London, 1938), p. 332.

K. Nicolaides, The Natural Way to Draw (Houghton Mifflin, Boston, Mass., 1941).

J. P. Hailman, “Environmental light and conspicuous colors,” in The Bahavioral Significance of Color, E. H. Burtt, ed. (Garland, New York, 1979).

R. Wehner, “Spatial vision in arthropods,” in Handbook of Sensory Physiology, H. Autrum, ed., Vol. C of Comparative Physiology and Evolution of Vision in Invertebrates (Springer-Verlag, Berlin, 1981).
[Crossref]

I. M. S. Langer, S. W. Zucker, “Qualitative shape from active shading,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Institute of Electrical and Electronics Engineers, New York, 1992), pp. 713–715.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1
Fig. 1

Image formed by overlapping a set of rectangles, each having a vertical intensity gradient. Most people perceive a set of flat surface patches protruding from a ground plane. A skyline and a graveyard are two typical interpretations. These percepts cannot be explained by classical shape-from-shading models, which require that smooth variations in image intensity depict smooth variations in surface normal. Classical models would interpret the tombstones as curved surfaces, such as cylinders!

Fig. 2
Fig. 2

Hemisphere of incident directions above a point on a surface, which can be partitioned into two sets. ν(x), shown by the unshaded region, denotes the set of directions in which the light source is visible from x. Its complement, νc(x), denotes the set of directions pointing to other surfaces in the scene.

Fig. 3
Fig. 3

(a) Surface luminance function. (b) Shadowing effects represented geometrically by the partition of the hemisphere of incident directions for each point. Notice that points in which less of the sky is visible tend to have lower luminance.

Fig. 4
Fig. 4

(a) 50 × 50 smooth surface. The depth values of the surface span a range of 0 to 25. (b), (c), (d) Surface rendered according to the radiosity equation for three values of the albedo (0.2, 0.5, 0.8, respectively). The uniform hemispheric source was approximated with 40 point sources. (e)–(g) Scatter plot of aperture versus luminance for each of the three albedos. The two solid curves represent the upper and lower bounds of the aperture–luminance inequality. These bounds are quite weak relative to the values found for a typical surface.

Fig. 5
Fig. 5

(a), (b), (c) Concave 50 × 50 Gaussian function having standard deviation 15 pixels and a maximum depth of 50, rendered according to the radiosity equation with three values of ρ (0.2,0.5,0.8, respectively). (d)–(f) Scatter plots of aperture versus luminance. (The leakage of the data beyond the bounds is a quantization effect.) Clearly, the average aperture for a given luminance varies with ρ. However, the variation of the aperture for a given luminance is roughly the same for all three values of ρ. This suggests that the variation in A(x) for a given value of I(x) is independent of ρ.

Fig. 6
Fig. 6

Two different quantizations of *, having 32 (left) and 64 (right) light-source directions, respectively. Each of these unit vectors points from the origin to a node in 3.

Fig. 7
Fig. 7

Example of a quantized visibility field for five light-source directions. Free nodes, *, are represented by open disks. Surface nodes, ∂, are represented by filled disks. Nodes below the surface are represented as points.

Fig. 8
Fig. 8

Shape-from-shading on a cloudy day algorithm applied to each of the three rendered images of Fig. 4. Each column corresponds to a single rendered image of Fig. 4. Each row above corresponds to a different chosen value of ρ (0.2,0.5,0.8). The three surfaces along the diagonal thus correspond to the correctly chosen values of ρ. See the text for a discussion of the errors.

Fig. 9
Fig. 9

Because z and * are quantized, the two algorithms–aperture from depth (see Appendix C) and depth from aperture–are not inverses of each other. When these two algorithms are run in succession on an original constant depth map, the resulting computed depth is not constant. (a) Mesh plot of a computed depth map for an original 50 × 50 depth map having constant depth (z ≡ 40) and with relatively coarse quantization parameters (M = 32, and z quantized to unit steps). (b) Mesh plot for a finer quantization (M = 64, and z quantized to half-unit steps).

Fig. 10
Fig. 10

Aperture from depth and depth from aperture algorithms were run in succession on a set of depth maps, each having constant depth. The mean and the standard deviation of the difference between the original and the computed depth map are plotted. (a) Coarse quantization (M = 32, and z quantized to unit steps), (b) fine quantization (M = 64, with z ^ quantized to half-unit steps). See the text for a discussion of the errors.

Fig. 11
Fig. 11

Scatter plot of luminance–depth pairs for the rendered images of Fig. 4: (a) ρ = 0.2, (b) ρ = 0.8. A darker-is-deeper heuristic is valid only in a loose statistical sense. The large variance of the relation implies that the heuristic is unreliable for recovering a depth map from an image.

Tables (1)

Tables Icon

Table 1 Mean Squared Depth Error of the Computed Surfaces of Fig. 8a

Equations (33)

Equations on this page are rendered with MathJax. Learn more.

N : F unit sphere .
H ( x ) { L : L unit sphere ,             and             L · N ( x ) > 0 } ,
I out ( x ) = ρ π H ( x ) I in ( x , L ) L · N ( x ) d Ω .
I in ( x , L ) I D .
I out ( x ) = ρ π I D ν ( x ) L · N ( x ) d Ω + ρ π ν c ( x ) I in ( x , L ) L · N ( x ) d Ω .
I in ( x , L ) = I out [ Π ( x , L ) ] .
I out ( x ) = ρ π I D ν ( x ) L · N ( x ) d Ω + ρ π ν c ( x ) I out [ Π ( x , L ) ] L · N ( x ) d Ω .
0 I out ( x ) ρ I D .
A ( x ) 1 2 π ν ( x ) d Ω .
A ( x ) 1 π ν ( x ) L · N ( x ) d Ω A ( x ) [ 2 - A ( x ) ] .
ρ π I D ν ( x ) L · N ( x ) d Ω I out ( x ) ρ 2 I D + ( 1 - ρ ) ρ π I D ν ( x ) L · N ( x ) d Ω .
ρ I D A ( x ) 2 I out ( x ) ρ 2 I D + ( 1 - ρ ) ρ I D A ( x ) [ 2 - A ( x ) ] ,
max ( 0 , 1 - { 1 1 - ρ [ 1 - 1 out ( x ) ρ I D ] } 1 / 2 ) A ( x ) [ I out ( x ) ρ I D ] 1 / 2 .
A ˜ ( x ) 1 2 [ [ I out ( x ) ρ I D ] 1 / 2 + max ( 0 , 1 - { 1 1 - ρ [ 1 - I out ( x ) ρ I D ] } 1 / 2 ) ] .
D ( x ) - ν ( x ) I D L d Ω .
A ( x ) 1 2 π ν ( x ) d Ω .
L 3 { x = ( x , y , n ) : x , y , n L } .
z : P { 0 , 1 , } .
F * { [ x , y , z ( x , y ) ] : ( x , y ) P } L 3 .
H * { L k : k = 1 , M } .
A * ( x ) ν * ( x ) H * .
ν n * ( x , y ) ν * ( x ) ,             A n * ( x , y ) A * ( x ) .
A ˜ * ( x ) 1 2 [ [ I out ( x ) I max * ] 1 / 2 + max ( 0 , 1 - { 1 1 - ρ [ 1 - I out ( x ) I max * ] } 1 / 2 ) ] .
I out ( x * ) = ρ π ν ( x * ) I D L · N ( x * ) d Ω + ρ π ν c ( x * ) I out ( x * , L ) L · N ( x ) d Ω < ρ π H ( x * ) I out ( x * ) L · N ( x * ) d Ω .
I out ( x ) = ρ π ν ( x ) I D L · N ( x ) d Ω + ρ π ν c ( x ) I out [ Π ( x , L ) ] L · N ( x ) d Ω ρ π H ( x ) I D L · N ( x ) d Ω = ρ I D ,
L ( θ , ϕ ) : [ 0 , 2 π ] [ 0 , / 2 π ] unit sphere ,
L ( θ , ϕ ) · N ( x ) = cos ϕ , d Ω = sin ϕ d ϕ d θ .
A ( x ) = 0 2 π 0 ϕ max sin ϕ d ϕ d θ = 0 2 π ϕ min π / 2 sin ϕ d ϕ d θ .
A ( x ) = 1 - cos ϕ max = cos ϕ min .
ν max { L ( θ , ϕ ) : 0 θ 2 π , 0 < ϕ < ϕ max } , ν min { L ( θ , ϕ ) : 0 θ 2 π , ϕ min < ϕ < / 2 π } .
1 π ν min L · N ( x ) d Ω 1 π ν ( x ) L · N ( x ) d Ω 1 π ν max L · N ( x ) d Ω .
1 π ν min L · N ( x ) d Ω = cos 2 ϕ min , 1 π ν max L · N ( x ) d Ω = 1 - cos 2 ϕ max .
A ( x ) 2 1 π ν ( x ) L · N ( x ) d Ω A ( x ) [ 2 - A ( x ) ] .

Metrics