Abstract

We introduce a new process of shape estimation through the matching of photometric-stereo images, which are monocular images obtained under different illuminations. If the illumination directions are not far apart, and if the imaged surface is smooth, so that a linear approximation to the reflectance map is applicable, the disparities produced by the matching process can be related to the depth function of the imaged surface through a differential equation whose approximate solution is easily found. We thus obtain a closed-form expression for surface depth, depending only on the coefficients of the linear-reflectance-map function. If those coefficients are not available, a simple iterative scheme still allows the recovery of depth, up to an overall scale factor.

© 1998 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. R. J. Woodham, “Photometric method for determining surface orientation from multiple images,” Opt. Eng. (Bellingham) 19, 139–144 (1980).
    [Crossref]
  2. L. B. Wolff, “Shape understanding from Lambertian photometric flow fields,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Ca., 1989), pp. 46–52.
  3. K. M. Lee, C.-C. J. Kuo, “Shape reconstruction from photometric stereo,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Ca., 1992), pp. 479–484.
  4. R. Zhang, P.-S. Tsai, M. Shah, “Photomotion,” Comput. Vis. Image Underst. 63, 221–231 (1996).
    [Crossref]
  5. A. Pentland, “Photometric motion,” IEEE Trans. Pattern Anal. Mach. Intell. 13, 879–890 (1991).
    [Crossref]
  6. B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intel. 17, 185–203 (1981).
    [Crossref]
  7. A. Pentland, “Linear shape from shading,” Int. J. Comput. Vis. 4, 153–162 (1990).
    [Crossref]
  8. F. B. Hildebrand, Advanced Calculus for Applications (Prentice-Hall, Englewood Cliffs, N.J., 1962).
  9. J. R. A. Torreão, E. Roe, “Microcanonical optimization applied to visual processing,” Phys. Lett. A 205, 377–382 (1995).
    [Crossref]
  10. S. T. Barnard, “A stochastic approach to stereo vision,” in Proceedings of the Fifth National Conference on Artificial Intelligence (MIT, Cambridge, Mass., 1986), pp. 676–680.
  11. K. Ikeuchi, “Determining a depth map using a dual photometric stereo,” Int. J. Robotics Res. 6, 15–31 (1987).
  12. G. Busettini, G. S. Masson, F. A. Miles, “A role for stereoscopic depth cues in the rapid visual stabilization of the eyes,” Nature (London) 380, 342–345 (1996).
    [Crossref]

1996 (2)

R. Zhang, P.-S. Tsai, M. Shah, “Photomotion,” Comput. Vis. Image Underst. 63, 221–231 (1996).
[Crossref]

G. Busettini, G. S. Masson, F. A. Miles, “A role for stereoscopic depth cues in the rapid visual stabilization of the eyes,” Nature (London) 380, 342–345 (1996).
[Crossref]

1995 (1)

J. R. A. Torreão, E. Roe, “Microcanonical optimization applied to visual processing,” Phys. Lett. A 205, 377–382 (1995).
[Crossref]

1991 (1)

A. Pentland, “Photometric motion,” IEEE Trans. Pattern Anal. Mach. Intell. 13, 879–890 (1991).
[Crossref]

1990 (1)

A. Pentland, “Linear shape from shading,” Int. J. Comput. Vis. 4, 153–162 (1990).
[Crossref]

1987 (1)

K. Ikeuchi, “Determining a depth map using a dual photometric stereo,” Int. J. Robotics Res. 6, 15–31 (1987).

1981 (1)

B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intel. 17, 185–203 (1981).
[Crossref]

1980 (1)

R. J. Woodham, “Photometric method for determining surface orientation from multiple images,” Opt. Eng. (Bellingham) 19, 139–144 (1980).
[Crossref]

Barnard, S. T.

S. T. Barnard, “A stochastic approach to stereo vision,” in Proceedings of the Fifth National Conference on Artificial Intelligence (MIT, Cambridge, Mass., 1986), pp. 676–680.

Busettini, G.

G. Busettini, G. S. Masson, F. A. Miles, “A role for stereoscopic depth cues in the rapid visual stabilization of the eyes,” Nature (London) 380, 342–345 (1996).
[Crossref]

Hildebrand, F. B.

F. B. Hildebrand, Advanced Calculus for Applications (Prentice-Hall, Englewood Cliffs, N.J., 1962).

Horn, B. K. P.

B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intel. 17, 185–203 (1981).
[Crossref]

Ikeuchi, K.

K. Ikeuchi, “Determining a depth map using a dual photometric stereo,” Int. J. Robotics Res. 6, 15–31 (1987).

Kuo, C.-C. J.

K. M. Lee, C.-C. J. Kuo, “Shape reconstruction from photometric stereo,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Ca., 1992), pp. 479–484.

Lee, K. M.

K. M. Lee, C.-C. J. Kuo, “Shape reconstruction from photometric stereo,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Ca., 1992), pp. 479–484.

Masson, G. S.

G. Busettini, G. S. Masson, F. A. Miles, “A role for stereoscopic depth cues in the rapid visual stabilization of the eyes,” Nature (London) 380, 342–345 (1996).
[Crossref]

Miles, F. A.

G. Busettini, G. S. Masson, F. A. Miles, “A role for stereoscopic depth cues in the rapid visual stabilization of the eyes,” Nature (London) 380, 342–345 (1996).
[Crossref]

Pentland, A.

A. Pentland, “Photometric motion,” IEEE Trans. Pattern Anal. Mach. Intell. 13, 879–890 (1991).
[Crossref]

A. Pentland, “Linear shape from shading,” Int. J. Comput. Vis. 4, 153–162 (1990).
[Crossref]

Roe, E.

J. R. A. Torreão, E. Roe, “Microcanonical optimization applied to visual processing,” Phys. Lett. A 205, 377–382 (1995).
[Crossref]

Schunck, B. G.

B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intel. 17, 185–203 (1981).
[Crossref]

Shah, M.

R. Zhang, P.-S. Tsai, M. Shah, “Photomotion,” Comput. Vis. Image Underst. 63, 221–231 (1996).
[Crossref]

Torreão, J. R. A.

J. R. A. Torreão, E. Roe, “Microcanonical optimization applied to visual processing,” Phys. Lett. A 205, 377–382 (1995).
[Crossref]

Tsai, P.-S.

R. Zhang, P.-S. Tsai, M. Shah, “Photomotion,” Comput. Vis. Image Underst. 63, 221–231 (1996).
[Crossref]

Wolff, L. B.

L. B. Wolff, “Shape understanding from Lambertian photometric flow fields,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Ca., 1989), pp. 46–52.

Woodham, R. J.

R. J. Woodham, “Photometric method for determining surface orientation from multiple images,” Opt. Eng. (Bellingham) 19, 139–144 (1980).
[Crossref]

Zhang, R.

R. Zhang, P.-S. Tsai, M. Shah, “Photomotion,” Comput. Vis. Image Underst. 63, 221–231 (1996).
[Crossref]

Artif. Intel. (1)

B. K. P. Horn, B. G. Schunck, “Determining optical flow,” Artif. Intel. 17, 185–203 (1981).
[Crossref]

Comput. Vis. Image Underst. (1)

R. Zhang, P.-S. Tsai, M. Shah, “Photomotion,” Comput. Vis. Image Underst. 63, 221–231 (1996).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

A. Pentland, “Photometric motion,” IEEE Trans. Pattern Anal. Mach. Intell. 13, 879–890 (1991).
[Crossref]

Int. J. Comput. Vis. (1)

A. Pentland, “Linear shape from shading,” Int. J. Comput. Vis. 4, 153–162 (1990).
[Crossref]

Int. J. Robotics Res. (1)

K. Ikeuchi, “Determining a depth map using a dual photometric stereo,” Int. J. Robotics Res. 6, 15–31 (1987).

Nature (London) (1)

G. Busettini, G. S. Masson, F. A. Miles, “A role for stereoscopic depth cues in the rapid visual stabilization of the eyes,” Nature (London) 380, 342–345 (1996).
[Crossref]

Opt. Eng. (Bellingham) (1)

R. J. Woodham, “Photometric method for determining surface orientation from multiple images,” Opt. Eng. (Bellingham) 19, 139–144 (1980).
[Crossref]

Phys. Lett. A (1)

J. R. A. Torreão, E. Roe, “Microcanonical optimization applied to visual processing,” Phys. Lett. A 205, 377–382 (1995).
[Crossref]

Other (4)

S. T. Barnard, “A stochastic approach to stereo vision,” in Proceedings of the Fifth National Conference on Artificial Intelligence (MIT, Cambridge, Mass., 1986), pp. 676–680.

F. B. Hildebrand, Advanced Calculus for Applications (Prentice-Hall, Englewood Cliffs, N.J., 1962).

L. B. Wolff, “Shape understanding from Lambertian photometric flow fields,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Ca., 1989), pp. 46–52.

K. M. Lee, C.-C. J. Kuo, “Shape reconstruction from photometric stereo,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE Computer Society Press, Los Alamitos, Ca., 1992), pp. 479–484.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1

Shape estimation through DBPS: reconstruction of z(x)=cos(x). Illumination angles (a) θ=1°, (b) θ=5°, and (c) θ=15°. Top, cos(x) (crosses) and estimated shape function (diamonds); center, error function δ1(x); bottom, error function δ2(x).

Fig. 2
Fig. 2

Shape estimation through DBPS: (a) and (b) input image pair, (c) estimated photometric disparity map, (d) depth map from Eq. (23), (e) depth map with intensity image overlay.

Fig. 3
Fig. 3

Shape estimation through DBPS: (a) and (b) input image pair, (c) estimated photometric disparity map, (d) depth map from Eq. (23), (e) depth map with intensity image overlay.

Fig. 4
Fig. 4

Shape estimation through DBPS: (a) and (b) input image pair, (c) estimated photometric disparity map, (d) depth map through iterative scheme of Eqs. (28) and (29), (e) depth map with intensity image overlay.

Fig. 5
Fig. 5

Shape estimation through DBPS: (a) and (b) input image pair, (c) estimated photometric disparity map, (d) depth map from Eq. (23), (e) depth map with intensity image overlay.

Fig. 6
Fig. 6

Shape estimation through DBPS: (a) and (b) input image pair, (c) estimated photometric disparity map, (d) depth map through iterative scheme of Eqs. (28) and (29), (e) depth map with intensity image overlay.

Fig. 7
Fig. 7

Shape estimation through DBPS: (a) and (b) input image pair, (c) estimated photometric disparity map, (d) depth map through iterative scheme of Eqs. (28) and (29), (e) depth map with intensity image overlay.

Equations (45)

Equations on this page are rendered with MathJax. Learn more.

I1(x, y)I2[x+Dx(s), y+Dy(s)],
ΔI(s)I1(x, y)-I2(x, y)Dx(s) I2x+Dy(s) I2y,
ΔI(s)=k0+k1p+k2q,
k¯0j=Rj(p0, q0),k1j=Rjp(p0, q0), k2j=Rjq(p0, q0),
k1p+k2qDx(s) I2x+Dy(s) I2y-k0,
nˆ=(-p,-q, 1)(p2+q2+1)1/2.
Dx(s)p+Dy(s)qv(s).
P(x, y)p+Q(x, y)q=R(x, y),
u1(x, y, z)=c1,u2(x, y, z)=c2,
dxP(x, y)=dyQ(x, y)=dzR(x, y),
u2(x, y, z)=F[u1(x, y, z)],
dxk1=dyk2=dzDx(s) I2x+Dy(s) I2y-k0,
dxDx(s)=dyDy(s)=dzv(s).
u1(x, y, z)k2x-k1y=c1,
Dy(s)Dx(s)=k2k1,
dzDx(s) I2x+Dy(s) I2y-k0=k1dx+k2dyk12+k22,
(k·ds)[D(s)·I2-k0]=(k12+k22)dz,
[k·D(s)]I2·ds-k0(k·ds)=(k12+k22)dz.
dff(x, y)·ds=(k12+k22)dz,
f(x, y)=[k·D(s)]I2(s)-k0(k1x+k2y),
u2(x, y, z)f(x, y)-(k12+k22)z=c2.
z(x, y)=[k·D(s)]I2(s)-k0(k1x+k2y)+F(k2x-k1y)k12+k22,
z(x, y)=Dx(s)I2(s)k1-k0(k1x+k2y)-F(k2x-k1y)k12+k22.
z(x, y)=Dx(s)I2(s)ρ sin σ cos τ-cos σ(x cos τ+y sin τ)sin σ.
z(x, y)=Dxst(s)+xsin σ-cos σ(x cos τ+y sin τ)sin σ
zk1z=DxI2-k0Dx(Dxx+Dyy)Dx2+Dy2,
k0=ΔIavg-pavg-DyDxqavg,
z(0)=DxI2-(ΔIavg) Dxx+DyyDx2+Dy2,
z(n)=z(0)+(Dxpavg(n-1)+Dyqavg(n-1))Dxx+DyyDx2+Dy2,
E({D(s)})=s|I1(s)-I2[s+D(s)]|+λsS(s),
I(x)nˆ·sˆ=sin(θ)sin(x)+cos(θ)[1+sin2(x)]1/2.
sin(θ)sin(x)+cos(θ)[1+sin2(x+Dx)]1/2=1[1+sin2(x)]1/2,
Dx=±arcsinsin(θ)cos(θ)[1+sin2(x)]±sin(x)1-sin2(θ)[1+sin2(x)]-x.
z(x)=Dxsin(θ) sin(θ)sin(x)+cos(θ)[1+sin2(x)]1/2-[1-cos(θ)]xsin(θ).
δ1(x)ΔI-k0-k1p=ΔI-k0+k1  sin(x),
δ2(x)ΔI-DxI2x,
ΔI=1-cos(θ)-sin(θ)sin(x)[1+sin2(x)]1/2.
ΔC(s)C1(s)-C2(s)=k0+k1p+k2q
ΔCi(j)=k0(i)+k1(i)pi(j)+k2(i)qi(j),j=1, 2,,n2,
K(i)=(βiTβi)-1βiTΔCi,
r=RσRτr,
Rσ=cos σ0sin σ010-sin σ0cos σ,
Rτ=cos τsin τ0-sin τcos τ0001
x=x cos σ cos τ+y cos σ sin τ+z sin σ,
y=-x sin τ+y cos τ.

Metrics