Abstract

Textured surface analysis is essential for many applications. We present a three-dimensional recovery approach for real textured surfaces based on photometric stereo. The aim is to be able to measure the textured surfaces with a high degree of accuracy. For this, we use a color digital sensor and principles of color photometric stereo. This method uses a single color image, instead of a sequence of gray-scale images, to recover the surface of the three dimensions. It can thus be integrated into dynamic systems where there is significant relative motion between the object and the camera. To evaluate the performance of our method, we compare it on real textured surfaces to traditional photometric stereo using three images. We thus show that it is possible to have similar results with just one color image.

© 2008 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |

  1. H. Zahouani, R. Vargiolu, and M.-T. Do, "Characterization of micro texture related to wet road/tire friction," in Proceedings of AIPCR/PIARC (AIPCR/PIARC, 2000), pp. 195-205.
  2. O. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint (MIT, 1995).
  3. B. Shahraray and M. Brown, "Robust depth estimation from optical flow," in Second International Conference on Computer Vision (IEEE, 1988), pp. 641-650.
  4. B. Horn, "Shape from shading: a method for obtaining the shape of a smooth opaque object from one view," Ph.D. dissertation (Massachusetts Institute of Technology, 1970).
  5. R. Zhang, P. Tsai, J. Cryer, and M. Shah, "Shape from shading: a survey," IEEE Trans. Pattern Anal. Mach. Intell. 21, 690-706 (1999).
    [CrossRef]
  6. R. Woodham, "Photometric method for determining surface orientation from multiple images," Opt. Eng. (Bellingham) 19, 139-144 (1980).
  7. J. E. N. Coleman and R. C. Jain, "Obtaining 3-dimensional shape of textured and specular surface using four-source photometry," Comput. Graph. Image Process. 18, 309-328 (1982).
    [CrossRef]
  8. K. Ikeuchi, "Determining surface orientations of specular surfaces by using the photometric stereo method," IEEE Trans. Pattern Anal. Mach. Intell. 3, 141-184 (1981).
    [CrossRef]
  9. G. McGunnigle and M. Chantler, "Rough surface description using photometric stereo," Meas. Sci. Technol. 14, 699-709 (2003).
    [CrossRef]
  10. A. Ben Slimane, M. Khoudeir, J. Brochard, V. Legeay, and M.-T. Do, "Relief reconstruction of rough textured surface through image analysis," Presented at the IS&T/SPIE 15th Annual Symposium on Electronic Imaging Science and Technology, Santa Clara, Calif., January 23-24, 2003.
  11. M. Khoudeir, J. Brochard, A. Benslimane, and M.-T. Do, "Estimation of the luminance map for a Lambertian photometric model: application to the study of road surface roughness," J. Electron. Imaging 3, 512-522 (2004).
  12. A. Pentland, "Photometric motion," IEEE Trans. Pattern Anal. Mach. Intell. 13, 879-890 (1991).
    [CrossRef]
  13. T. Malzbender, B. Wilburn, D. Geld, and B. Ambrisco, "Surface enhancement using real-time photometric stereo and reflectance transformation," presented at the Eurographics Symposium on Rendering, Nicosia, Cyprus, June 26-28, 2006.
  14. M. S. Drew, "Shape from color," Tech. Rep. CSS/LCCR TR 97-07 (Centre for Scientific Computing, 1992).
  15. M. L. Smith and L. N. Smith, "Dynamic photometric stereo--a new technique for moving surface analysis," Image Vis. Comput. 23841-852 (2005).
    [CrossRef]
  16. M. S. Drew, "Photometric stereo without multiple images," Proc. SPIE 3016, 369-380 (1997).
    [CrossRef]
  17. G. Sharma, Digital Color Imaging Handbook (CRC, 2003).
  18. J. Lambert, Photometria (Augsburg, 1760).
  19. B. Horn, "Height and gradient from shading," Int. J. Comput. Vis. 5, 37-75 (1990).
    [CrossRef]
  20. R. Klette and K. Schluns, "Height data from gradient fields," Proc. SPIE 2008, 204-215 (1996).
    [CrossRef]
  21. T. Wei and R. Klette, "Height from gradient using surface curvature and area constraints," in Third Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP) (ICVGIP, 2002), pp. 204-210.
  22. P. Kovesi, "Shapelets correlated with surface normals produce surfaces," in IEEE International Conference on Computer Vision (IEEE, 2005), Vol. 2, pp. 994-1001.
  23. A. Woodward and P. Delmas, "Synthetic ground truth for comparison of gradient field integration methods for human faces," presented at the Image and Vision Computing New Zealand Conference, Dunedin, New Zealand, November 28-29, 2005.
  24. A. Agrawal, R. Raskar, and R. Chellappa, "What is the range of surface reconstructions from a gradient field?" in European Conference on Computer Vision (ECCV) (Springer, 2006), pp. 578-591.
  25. R. Frankot and R. Chellappa, "A method for enforcing integrability in shape from shading algorithms," IEEE Trans. Pattern Anal. Mach. Intell. 10, 439-451 (1988).
    [CrossRef]
  26. K. Schlüns and R. Klette, "Local and global integration of discrete vector fields," in Advances in Computer Vision, F.Solina, W.G.Kropatsch, R.Klette, and R.Bajesy, eds. (Springer, 1997), pp. 149-158.
    [CrossRef]
  27. J. Y. Hardeberg, Acquisition and Reproduction of Color Images: Colorimetric and Multispectral Approaches (Universal, 2001).
  28. X. Huang, J. Brochard, D. Helbert, and M. Khoudeir, "Relief extraction of rough textured reflecting surface by image processing," presented at the IEEE/SPIE International Conference on Quality Control by Artificial Vision, Le Creusot, France, May 23-25, 2007.
  29. M. Oren and S. Nayar, "Generalization of the Lambertian model and implications for machine vision," Int. J. Comput. Vis. 14, 227-251 (1995).
    [CrossRef]
  30. D. Helbert, M. Khoudeir, and M.-T. Do, "Rough surfaces and relief extraction by generalized Lambertian's photometric model," presented at the IEEE International Conference on Signal Processing and Communication, Dubai, United Arab Emirates, November 24-27, 2007.

2005 (1)

M. L. Smith and L. N. Smith, "Dynamic photometric stereo--a new technique for moving surface analysis," Image Vis. Comput. 23841-852 (2005).
[CrossRef]

2004 (1)

M. Khoudeir, J. Brochard, A. Benslimane, and M.-T. Do, "Estimation of the luminance map for a Lambertian photometric model: application to the study of road surface roughness," J. Electron. Imaging 3, 512-522 (2004).

2003 (1)

G. McGunnigle and M. Chantler, "Rough surface description using photometric stereo," Meas. Sci. Technol. 14, 699-709 (2003).
[CrossRef]

1999 (1)

R. Zhang, P. Tsai, J. Cryer, and M. Shah, "Shape from shading: a survey," IEEE Trans. Pattern Anal. Mach. Intell. 21, 690-706 (1999).
[CrossRef]

1997 (1)

M. S. Drew, "Photometric stereo without multiple images," Proc. SPIE 3016, 369-380 (1997).
[CrossRef]

1996 (1)

R. Klette and K. Schluns, "Height data from gradient fields," Proc. SPIE 2008, 204-215 (1996).
[CrossRef]

1995 (1)

M. Oren and S. Nayar, "Generalization of the Lambertian model and implications for machine vision," Int. J. Comput. Vis. 14, 227-251 (1995).
[CrossRef]

1991 (1)

A. Pentland, "Photometric motion," IEEE Trans. Pattern Anal. Mach. Intell. 13, 879-890 (1991).
[CrossRef]

1990 (1)

B. Horn, "Height and gradient from shading," Int. J. Comput. Vis. 5, 37-75 (1990).
[CrossRef]

1988 (1)

R. Frankot and R. Chellappa, "A method for enforcing integrability in shape from shading algorithms," IEEE Trans. Pattern Anal. Mach. Intell. 10, 439-451 (1988).
[CrossRef]

1982 (1)

J. E. N. Coleman and R. C. Jain, "Obtaining 3-dimensional shape of textured and specular surface using four-source photometry," Comput. Graph. Image Process. 18, 309-328 (1982).
[CrossRef]

1981 (1)

K. Ikeuchi, "Determining surface orientations of specular surfaces by using the photometric stereo method," IEEE Trans. Pattern Anal. Mach. Intell. 3, 141-184 (1981).
[CrossRef]

1980 (1)

R. Woodham, "Photometric method for determining surface orientation from multiple images," Opt. Eng. (Bellingham) 19, 139-144 (1980).

Comput. Graph. Image Process. (1)

J. E. N. Coleman and R. C. Jain, "Obtaining 3-dimensional shape of textured and specular surface using four-source photometry," Comput. Graph. Image Process. 18, 309-328 (1982).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (4)

K. Ikeuchi, "Determining surface orientations of specular surfaces by using the photometric stereo method," IEEE Trans. Pattern Anal. Mach. Intell. 3, 141-184 (1981).
[CrossRef]

R. Zhang, P. Tsai, J. Cryer, and M. Shah, "Shape from shading: a survey," IEEE Trans. Pattern Anal. Mach. Intell. 21, 690-706 (1999).
[CrossRef]

A. Pentland, "Photometric motion," IEEE Trans. Pattern Anal. Mach. Intell. 13, 879-890 (1991).
[CrossRef]

R. Frankot and R. Chellappa, "A method for enforcing integrability in shape from shading algorithms," IEEE Trans. Pattern Anal. Mach. Intell. 10, 439-451 (1988).
[CrossRef]

Image Vis. Comput. (1)

M. L. Smith and L. N. Smith, "Dynamic photometric stereo--a new technique for moving surface analysis," Image Vis. Comput. 23841-852 (2005).
[CrossRef]

Int. J. Comput. Vis. (2)

B. Horn, "Height and gradient from shading," Int. J. Comput. Vis. 5, 37-75 (1990).
[CrossRef]

M. Oren and S. Nayar, "Generalization of the Lambertian model and implications for machine vision," Int. J. Comput. Vis. 14, 227-251 (1995).
[CrossRef]

J. Electron. Imaging (1)

M. Khoudeir, J. Brochard, A. Benslimane, and M.-T. Do, "Estimation of the luminance map for a Lambertian photometric model: application to the study of road surface roughness," J. Electron. Imaging 3, 512-522 (2004).

Meas. Sci. Technol. (1)

G. McGunnigle and M. Chantler, "Rough surface description using photometric stereo," Meas. Sci. Technol. 14, 699-709 (2003).
[CrossRef]

Opt. Eng. (Bellingham) (1)

R. Woodham, "Photometric method for determining surface orientation from multiple images," Opt. Eng. (Bellingham) 19, 139-144 (1980).

Proc. SPIE (2)

M. S. Drew, "Photometric stereo without multiple images," Proc. SPIE 3016, 369-380 (1997).
[CrossRef]

R. Klette and K. Schluns, "Height data from gradient fields," Proc. SPIE 2008, 204-215 (1996).
[CrossRef]

Other (17)

T. Wei and R. Klette, "Height from gradient using surface curvature and area constraints," in Third Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP) (ICVGIP, 2002), pp. 204-210.

P. Kovesi, "Shapelets correlated with surface normals produce surfaces," in IEEE International Conference on Computer Vision (IEEE, 2005), Vol. 2, pp. 994-1001.

A. Woodward and P. Delmas, "Synthetic ground truth for comparison of gradient field integration methods for human faces," presented at the Image and Vision Computing New Zealand Conference, Dunedin, New Zealand, November 28-29, 2005.

A. Agrawal, R. Raskar, and R. Chellappa, "What is the range of surface reconstructions from a gradient field?" in European Conference on Computer Vision (ECCV) (Springer, 2006), pp. 578-591.

K. Schlüns and R. Klette, "Local and global integration of discrete vector fields," in Advances in Computer Vision, F.Solina, W.G.Kropatsch, R.Klette, and R.Bajesy, eds. (Springer, 1997), pp. 149-158.
[CrossRef]

J. Y. Hardeberg, Acquisition and Reproduction of Color Images: Colorimetric and Multispectral Approaches (Universal, 2001).

X. Huang, J. Brochard, D. Helbert, and M. Khoudeir, "Relief extraction of rough textured reflecting surface by image processing," presented at the IEEE/SPIE International Conference on Quality Control by Artificial Vision, Le Creusot, France, May 23-25, 2007.

D. Helbert, M. Khoudeir, and M.-T. Do, "Rough surfaces and relief extraction by generalized Lambertian's photometric model," presented at the IEEE International Conference on Signal Processing and Communication, Dubai, United Arab Emirates, November 24-27, 2007.

G. Sharma, Digital Color Imaging Handbook (CRC, 2003).

J. Lambert, Photometria (Augsburg, 1760).

T. Malzbender, B. Wilburn, D. Geld, and B. Ambrisco, "Surface enhancement using real-time photometric stereo and reflectance transformation," presented at the Eurographics Symposium on Rendering, Nicosia, Cyprus, June 26-28, 2006.

M. S. Drew, "Shape from color," Tech. Rep. CSS/LCCR TR 97-07 (Centre for Scientific Computing, 1992).

A. Ben Slimane, M. Khoudeir, J. Brochard, V. Legeay, and M.-T. Do, "Relief reconstruction of rough textured surface through image analysis," Presented at the IS&T/SPIE 15th Annual Symposium on Electronic Imaging Science and Technology, Santa Clara, Calif., January 23-24, 2003.

H. Zahouani, R. Vargiolu, and M.-T. Do, "Characterization of micro texture related to wet road/tire friction," in Proceedings of AIPCR/PIARC (AIPCR/PIARC, 2000), pp. 195-205.

O. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint (MIT, 1995).

B. Shahraray and M. Brown, "Robust depth estimation from optical flow," in Second International Conference on Computer Vision (IEEE, 1988), pp. 641-650.

B. Horn, "Shape from shading: a method for obtaining the shape of a smooth opaque object from one view," Ph.D. dissertation (Massachusetts Institute of Technology, 1970).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1
Fig. 1

Geometric definition of angles σ and τ.

Fig. 2
Fig. 2

Example of a Bayer filter on a numerical camera.

Fig. 3
Fig. 3

Example of a computer-created half-sphere lighted by three isolated colored sources of isolated light: a red one at τ 1 = π 3 (bottom right), a green one at τ 2 = π (top), and a blue one at τ 3 = 5 π 3 (bottom left).

Fig. 4
Fig. 4

Color photometric stereo: Three white lights with three different spectral filters illuminate the textured surface. For one facet, the reflected spectral radiance distribution depends on the normal direction. Then the Bayer filter of the camera produces three isolated spectral images. Finally, the 3D surface can be recovered with these images and the photometric stereo theory.

Fig. 5
Fig. 5

Three-dimensional surfaces acquired by a laser system.

Fig. 6
Fig. 6

Framework of the simulation.

Fig. 7
Fig. 7

Simulation of acquired images of a 3D textured surface illuminated according to Lambert’s model. The zenith angle σ is equal to 3 π 70 : (a) 3D representation, (b) τ 1 = π 3 , (c) τ 2 = π , and (d) τ 3 = 5 π 3 .

Fig. 8
Fig. 8

Simulation of acquired images of a 3D textured surface illuminated by three colored lights with different azimuth angles (the blue light, τ b = π 3 ; the green light, τ g = π ; and the red light, τ r = 5 π 3 ) with Lambert’s model. The zenith angle σ is equal to 3 π 70 : (a) 3D representation and (b) color image.

Fig. 9
Fig. 9

Example of a part of the original textured surface (Fig. 5) and the same part of the reconstructed surface: (a) original surface, (b) reconstructed surface, and (c) error map.

Fig. 10
Fig. 10

Illustration of noise addition for three gray-level image acquisition (a) or one color image acquisition (b). In the case of three acquisitions of gray-level images: only the azimuth angle τ = π is represented. The SNR is equal to 8.105.

Fig. 11
Fig. 11

Acquisition system with the color lights.

Fig. 12
Fig. 12

Spectral analysis of the 3D reconstruction system: (a) gas-discharge lamp, (b) color filters, (c) Bayer filter, and (d) signal received.

Fig. 13
Fig. 13

Example of two real acquisition surfaces: (a) road surface and (b) stucco.

Fig. 14
Fig. 14

Comparison between color photometric reconstruction and standard photometric reconstruction for the surface in Fig. 13: (a) three images, (b) color image, (c) zoom for three images, and (d) zoom for one color image.

Fig. 15
Fig. 15

Comparison between color photometric reconstruction and standard photometric reconstruction for the surface in Fig. 13: (a) three images, (b) color image, (c) zoom for three images, and (d) zoom for one color image.

Tables (2)

Tables Icon

Table 1 SNR Error (dB) between the Original Surface and the Recovered Surface from Three Images or One Color Image

Tables Icon

Table 2 Evolution of SNR between the Reconstruction without Noise and with Noise as a Function of the Noise Variance for the Surface in Fig. 5a, in the Case of Three Gray-Level Images or One Color Image

Equations (15)

Equations on this page are rendered with MathJax. Learn more.

I = I 0 ρ cos ( θ ) = I 0 ρ ( N L ) ,
i ( x , y ) = i 0 ρ ( x , y ) p cos τ sin σ q sin τ sin σ + cos σ p 2 + q 2 + 1 ,
i k ( x , y ) = ρ ( x , y ) i 0 p S k x + q S k y S k z p 2 + q 2 + 1 with k = 1 , 2 , 3 ,
W = ( s ( x , y ) x p ) 2 + ( s ( x , y ) y q ) 2 d x d y ,
S ( u , v ) = j u P ( u , v ) j v Q ( u , v ) u 2 + v 2 ,
I ( λ ) = I 0 ( λ ) ρ ( λ ) .
I ( λ ) = I 0 ( λ ) ρ ( λ ) ( N L ) .
I c ( λ ) = I 0 ( λ ) ρ ( λ ) F c ( λ ) ( N L c ) ,
{ I 1 ( λ ) = I r , g , b C r ( λ ) I 2 ( λ ) = I r , g , b C g ( λ ) I 3 ( λ ) = I r , g , b C b ( λ ) } ,
I r , g , b = I 0 ( λ ) ρ ( λ ) × ( F r ( λ ) ( N L r ) + F g ( λ ) ( N L g ) + F b ( λ ) ( N L b ) ) ,
SNR = 10 log ( ( 1 m n ) i = 0 m 1 i = 0 m 1 S ( i , j ) 2 MSE ) ,
MSE = 1 m n i = 0 m 1 i = 0 m 1 S ( i , j ) S ̂ ( i , j ) 2 .
[ I 1 I 2 I 3 ] = I 0 ρ × Γ × [ ( N L r ) ( N L g ) ( N L b ) ] ,
Γ = [ C r × F r C r × F g C r × F b C g × F r C g × F g C g × F b C b × F r C b × F g C b × F b ] .
Γ = [ 1.000 0.236 0.042 0.073 1.000 0.139 0.058 0.033 1.000 ] .

Metrics