Abstract

We address the visual ambiguities that arise in estimating object and scene structure from a set of images when the viewpoint and lighting are unknown. We obtain a novel viewpoint–lighting ambiguity called the KGBR that corresponds to a group of three-dimensional affine transformations on the object or scene geometry combined with transformations on the object or scene albedo. Our analysis assumes orthographic projection with an affine camera model. We include photometric cues, such as shadowing and shading, that we model using Lambertian reflectance functions with shadows (cast and attached) and multiple light sources (but no interreflections). We relate the KGBR to affine ambiguities in estimating shape and to the generalized bas-relief (GBR) ambiguity.

© 2003 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. F. Phillips, J. T. Todd, J. J. Koenderink, A. M. L. Kappers, “Perceptual localization of surface position,” J. Exp. Psychol. Hum. Percep. Perform. 23, 1481–1492 (1997).
    [CrossRef]
  2. J. J. Koenderink, A. Van Doorn, “Affine structure from motion,” J. Opt. Soc. Am. A 8, 377–385 (1991).
    [CrossRef] [PubMed]
  3. O. Faugeras, “Stratification of three-dimensional vision: projective, affine, and metric representations,” J. Opt. Soc. Am. A 12, 465–484 (1995).
    [CrossRef]
  4. R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge U. Press, Cambridge, UK, 2000).
  5. P. N. Belhumeur, D. Kriegman, A. L. Yuille, “The generalized bas relief ambiguity,” Int. J. Comput. Vision 35, 33–44 (1999).
    [CrossRef]
  6. A. L. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
    [CrossRef]
  7. R. Rosenholtz, J. Koenderink, “Affine structure and photometry,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (IEEE Computer Society Press, Los Alamitos, Calif., 1996), pp. 760–795.
  8. A. L. Yuille, J. M. Coughlan, S. Konishi, “The KGBR viewpoint–lighting ambiguity and its resolution by generic constraints,” in Proceedings of the International Conference of Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif.2001), pp. 376–382.
  9. M. Werman, D. Weinshall, “Similarity and affine invariant distance between point sets,” IEEE Trans. Pattern Anal. Mach. Intell. 17, 810–814 (1995).
    [CrossRef]
  10. G. Strang, Introduction to Applied Mathematics (Wellesley-Cambridge Press, Wellesley, Mass., 1986).

1999 (2)

P. N. Belhumeur, D. Kriegman, A. L. Yuille, “The generalized bas relief ambiguity,” Int. J. Comput. Vision 35, 33–44 (1999).
[CrossRef]

A. L. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

1997 (1)

F. Phillips, J. T. Todd, J. J. Koenderink, A. M. L. Kappers, “Perceptual localization of surface position,” J. Exp. Psychol. Hum. Percep. Perform. 23, 1481–1492 (1997).
[CrossRef]

1995 (2)

O. Faugeras, “Stratification of three-dimensional vision: projective, affine, and metric representations,” J. Opt. Soc. Am. A 12, 465–484 (1995).
[CrossRef]

M. Werman, D. Weinshall, “Similarity and affine invariant distance between point sets,” IEEE Trans. Pattern Anal. Mach. Intell. 17, 810–814 (1995).
[CrossRef]

1991 (1)

Belhumeur, P.

A. L. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

Belhumeur, P. N.

P. N. Belhumeur, D. Kriegman, A. L. Yuille, “The generalized bas relief ambiguity,” Int. J. Comput. Vision 35, 33–44 (1999).
[CrossRef]

Coughlan, J. M.

A. L. Yuille, J. M. Coughlan, S. Konishi, “The KGBR viewpoint–lighting ambiguity and its resolution by generic constraints,” in Proceedings of the International Conference of Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif.2001), pp. 376–382.

Epstein, R.

A. L. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

Faugeras, O.

Hartley, R.

R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge U. Press, Cambridge, UK, 2000).

Kappers, A. M. L.

F. Phillips, J. T. Todd, J. J. Koenderink, A. M. L. Kappers, “Perceptual localization of surface position,” J. Exp. Psychol. Hum. Percep. Perform. 23, 1481–1492 (1997).
[CrossRef]

Koenderink, J.

R. Rosenholtz, J. Koenderink, “Affine structure and photometry,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (IEEE Computer Society Press, Los Alamitos, Calif., 1996), pp. 760–795.

Koenderink, J. J.

F. Phillips, J. T. Todd, J. J. Koenderink, A. M. L. Kappers, “Perceptual localization of surface position,” J. Exp. Psychol. Hum. Percep. Perform. 23, 1481–1492 (1997).
[CrossRef]

J. J. Koenderink, A. Van Doorn, “Affine structure from motion,” J. Opt. Soc. Am. A 8, 377–385 (1991).
[CrossRef] [PubMed]

Konishi, S.

A. L. Yuille, J. M. Coughlan, S. Konishi, “The KGBR viewpoint–lighting ambiguity and its resolution by generic constraints,” in Proceedings of the International Conference of Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif.2001), pp. 376–382.

Kriegman, D.

P. N. Belhumeur, D. Kriegman, A. L. Yuille, “The generalized bas relief ambiguity,” Int. J. Comput. Vision 35, 33–44 (1999).
[CrossRef]

Phillips, F.

F. Phillips, J. T. Todd, J. J. Koenderink, A. M. L. Kappers, “Perceptual localization of surface position,” J. Exp. Psychol. Hum. Percep. Perform. 23, 1481–1492 (1997).
[CrossRef]

Rosenholtz, R.

R. Rosenholtz, J. Koenderink, “Affine structure and photometry,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (IEEE Computer Society Press, Los Alamitos, Calif., 1996), pp. 760–795.

Snow, D.

A. L. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

Strang, G.

G. Strang, Introduction to Applied Mathematics (Wellesley-Cambridge Press, Wellesley, Mass., 1986).

Todd, J. T.

F. Phillips, J. T. Todd, J. J. Koenderink, A. M. L. Kappers, “Perceptual localization of surface position,” J. Exp. Psychol. Hum. Percep. Perform. 23, 1481–1492 (1997).
[CrossRef]

Van Doorn, A.

Weinshall, D.

M. Werman, D. Weinshall, “Similarity and affine invariant distance between point sets,” IEEE Trans. Pattern Anal. Mach. Intell. 17, 810–814 (1995).
[CrossRef]

Werman, M.

M. Werman, D. Weinshall, “Similarity and affine invariant distance between point sets,” IEEE Trans. Pattern Anal. Mach. Intell. 17, 810–814 (1995).
[CrossRef]

Yuille, A. L.

P. N. Belhumeur, D. Kriegman, A. L. Yuille, “The generalized bas relief ambiguity,” Int. J. Comput. Vision 35, 33–44 (1999).
[CrossRef]

A. L. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

A. L. Yuille, J. M. Coughlan, S. Konishi, “The KGBR viewpoint–lighting ambiguity and its resolution by generic constraints,” in Proceedings of the International Conference of Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif.2001), pp. 376–382.

Zisserman, A.

R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge U. Press, Cambridge, UK, 2000).

IEEE Trans. Pattern Anal. Mach. Intell. (1)

M. Werman, D. Weinshall, “Similarity and affine invariant distance between point sets,” IEEE Trans. Pattern Anal. Mach. Intell. 17, 810–814 (1995).
[CrossRef]

Int. J. Comput. Vision (2)

P. N. Belhumeur, D. Kriegman, A. L. Yuille, “The generalized bas relief ambiguity,” Int. J. Comput. Vision 35, 33–44 (1999).
[CrossRef]

A. L. Yuille, D. Snow, R. Epstein, P. Belhumeur, “Determining generative models for objects under varying illumination: shape and albedo from multiple images using SVD and integrability,” Int. J. Comput. Vision 35, 203–222 (1999).
[CrossRef]

J. Exp. Psychol. Hum. Percep. Perform. (1)

F. Phillips, J. T. Todd, J. J. Koenderink, A. M. L. Kappers, “Perceptual localization of surface position,” J. Exp. Psychol. Hum. Percep. Perform. 23, 1481–1492 (1997).
[CrossRef]

J. Opt. Soc. Am. A (2)

Other (4)

G. Strang, Introduction to Applied Mathematics (Wellesley-Cambridge Press, Wellesley, Mass., 1986).

R. Rosenholtz, J. Koenderink, “Affine structure and photometry,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (IEEE Computer Society Press, Los Alamitos, Calif., 1996), pp. 760–795.

A. L. Yuille, J. M. Coughlan, S. Konishi, “The KGBR viewpoint–lighting ambiguity and its resolution by generic constraints,” in Proceedings of the International Conference of Computer Vision (IEEE Computer Society Press, Los Alamitos, Calif.2001), pp. 376–382.

R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge U. Press, Cambridge, UK, 2000).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

Left panel: convex versus concave ambiguity. A convex object lit from above looks like a convex object lit from below. Right panel: the bas-relief ambiguity. The perception of shape is relatively insensitive to a linear scaling in the viewing direction.

Fig. 2
Fig. 2

Cube viewed from direction (0.51, 0.63, 0.58) (far-left panel) and the same cube undergoing affine transformations (remaining panels) seen from the same viewpoint.

Fig. 3
Fig. 3

If the lighting conditions are unknown, then it is impossible to distinguish between two objects related by a GBR transform.5 For any image of the first object, under one illumination condition, we can always find a corresponding illumination condition that makes the second object appear identical (i.e., generate an identical image). We show two objects under three different, but corresponding, lighting conditions.

Fig. 4
Fig. 4

We define intrinsic coordinates (u, v) on the surface of the object.

Fig. 5
Fig. 5

The cast shadow boundaries, and hence the cast shadows, are preserved by the KGBR. Similar results were shown for the GBR.5

Fig. 6
Fig. 6

For the orthographic camera (left) the vectors v, c1, c2 are orthogonal unit vectors. For the affine camera (right) c1, c2 are only constrained to be orthogonal to the unit vector v.

Fig. 7
Fig. 7

Joint viewpoint–lighting ambiguity. If two objects are related by a KGBR, then for any view of one there is a corresponding view of the other that is identical (after adjusting the setting of the affine camera) or identical up to a two-dimensional affine warp (for orthographic projection). The lighting is also transformed by the corresponding KGBR.

Fig. 8
Fig. 8

Top row, original object; bottom row, object after a KGBR (see text). Left panels, the objects look the same when viewed from direction (1, 0, 0) up to an affine warp on the images. But they look very different when viewed from direction (1/2, 0, 1/2) (center panels) and direction (1/2, 1/2, 1/2) (right panel).

Fig. 9
Fig. 9

Top row, original object (with albedo); bottom row, object after a KGBR (see text). Left panels, the objects look the same when viewed from direction (1, 0, 0) up to an affine warp on the images. But they look very different when viewed from direction (1/2, 0, 1/2) (center panels), and direction (1/2, 1/2, 1/2) (right panel).

Equations (29)

Equations on this page are rendered with MathJax. Learn more.

rˆ(u, v)=Kr(u, v),nˆ(u, v)=K-1,Tn(u, v)|K-1,Tn(u, v)|,
aˆ(u, v)=a(u, v)|Kj=1MSj||K-1,Tn(u, v)|,
sˆi=1|Kj=1mSj|Ksi·i=1,, m.
max{aˆ(u, v)nˆ(u, v)·sˆj,0}=max{a(u, v)n(u, v)·sj,0}j=1,, m,
p1(u, v)=c1·r(u, v),p2(u, v)=c2·r(u, v).
cˆ1=K-1,Tc1,cˆ2=K-1,Tc2,vˆ=Kv|Kv|.
cˆ1=A11K-1,Tc1+A12K-1,Tc2,
cˆ2=A21K-1,Tc1+A22K-1,Tc2,vˆ=Kv|Kv|,
A2=A11A12A21A22
cˆ1·rˆ(u, v)cˆ2·rˆ(u, v)=A11A12A21A22c1·r(u, v)c2·r(u, v).
10-μ/λ01-ν/λ001/λ
c1*·r(u, v)=c1*·Kr(u, v),u, v,c2*·r(u, v)=c2*·Kr(u, v),u, v.
K=100010μνλ,K-1,T=10-μ/λ01-ν/λ001/λ,
A3=A2001,
A2=A11A12A21A22
K-1,Tc1=B11cˆ1+B12cˆ2,K-1,Tc2=B21cˆ1+B22cˆ2,
B2=B11B12B21B22
A2=A11A12A21A22.
cˆ1=Φc1,cˆ2=Φc2,vˆ=Φv.
K-1,Tc1=Φ(B11c1+B12c2),K-1,Tc2=Φ(B21c1+B22c2).
K-1,T=ΦB11B21αB12B22β00γ=ΦGB3,
G=10α01β00γ
B3=B11B120B21B220001.
G-1,T=1000101.21.34,
A2=1002.
x1(u, v)=c1Tr(u, v),y1(u, v)=c2Tr(u, v),I1(u, v)=max{a(u, v)nT(u, v)s, 0},x2(u, v)=c1TMTr(u, v),y2(u, v)=c2TMTr(u, v),I1(u, v)=max{a(u, v)nT(u, v)s, 0}.
xˆ1(u, v)=cˆ1TKr(u, v),yˆ1(u, v)=cˆ2TKr(u, v),Iˆ1(u, v)=max{a(u, v)nT(u, v)s, 0},xˆ2(u, v)=cˆ1TNTKr(u, v),yˆ2(u, v)=cˆ2TNTKr(u, v),Iˆ1(u, v)=max{a(u, v)nT(u, v)s, 0}.
x2(u, v)=c1TMr(u, v),y2(u, v)=c2TMr(u, v),I2(u, v)=maxa(u, v)nT(u, v)M-1s|M-1,Tn(u, v)|, 0.
xˆ2(u, v)=cˆ1TNKr(u, v),yˆ2(u, v)=cˆ2TNKr(u, v),Iˆ2(u, v)=maxa(u, v)|K-1,Tn(u, v)|nT(u, v)K-1N-1Ks|N-1,TK-1,Tn(u, v)|, 0.

Metrics