Abstract

The correspondence problem of two captured images, which are obtained by projecting a structured light on the measuring surface, are explored for when three-dimensional information of a given surface is needed. In our system the constraint that codifies the pattern projected on the surface has been simplified by using a random speckle pattern, thus the correspondence problem is reduced to local matching between two captured images and solved by a spatial distance computation technique. The performance of our approach, which includes a disparity error analysis, a search range suggestion, and a disparity gradient limit, are investigated and discussed. Some parameters, such as percentile constraint, sampling interval, and subpixel compensation proper for use in this approach are suggested. Experiments have shown the feasibility of the proposed method.

© 2003 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Batlle, E. Mouaddib, J. Salvi, “Recent progress in coded structured light as a technique to solve the correspondence problem: a survey,” Pattern Recogn. 31, 963–982 (1998).
    [CrossRef]
  2. J. Salvi, J. Batlle, E. Mouaddib, “A robust-coded pattern projection for dynamic 3D scene measurement,” Pattern Recogn. Lett. 19, 1055–1065 (1998).
    [CrossRef]
  3. M. Zhou, C. S. Fraser, “Automated surface extraction in real time photogrammetry,” in XIXth ISPRS Congress, Proc. IAPRS XXXIII, Amsterdam, Netherlands, 943–950 (2000).
  4. J. Gühring, “Dense 3-D surface acquisition by structured light using off-the-shelf components,” in Photonics West 2001: Videometrics VII, Proc. SPIE4309, San Jose, Calif.220–231 (2001).
  5. C. Rocchini, P. Cignoni, C. Montani, P. Pingi, R. Scopigno, “A low cost 3D scanner based on structured light,” in Computer Graphics Forum, Proc. EUROGRAPHICS 20, 299–308 (2001).
  6. J. P. Siebert, S. J. Marshall, “Human body 3D imaging by speckle texture projection photogrammetry,” Sensor Review 20, 218–226 (2000).
    [CrossRef]
  7. N. D’Apuzzo, “Automated photogrammetric measurement of human faces,” International Archives of Photogrammetry and Remote Sensing32, Hakodate, Japan, 402–407 (1998).
  8. N. D’Apuzzo, “Human face modeling from multi images,” Proc. 3rd International Image Sensing Seminar on New Development in Digital Photogrammetry, Gifu, Japan, 28–29 (2001).
  9. N. D’Apuzzo, “Modeling human faces with multi-image photogrammetry,” in Three-Dimensional Image Capture and Applications V, Proc. SPIE4661, San Jose, Calif., 191–197 (2002).
  10. H. T. E. Hertzberg, C. W. Dupertuis, “Stereophotogrammetry as an anthropometric tool,” Photogrammetric Engineering 32, 942–947 (1957).
  11. H. K. Nishihara, “Practical real-time imaging stereo matcher,” Optical Eng. 23, 536–545 (1984).
    [CrossRef]
  12. A. Grün, “Adaptive least squares correlation: a powerful image matching technique,” South African Journal of Photogrammetry, Remote Sensing and Cartography14, 175–187 (1985).
  13. Y. S. Chen, Y. C. Hsu, “Image segmentation of a color-blindness plate,” Appl. Opt. 33, 6818–6822 (1994).
    [CrossRef] [PubMed]
  14. Y. S. Chen, Y. C. Hsu, “Computer vision on a colour blindness plate,” Image Vision Comput. 13, 463–478 (1995).
    [CrossRef]
  15. Y. S. Chen, M. H. Wang, “An approach to perceiving the multi-meaningful-dotted-pattern in a CBP image,” IEICE Trans. Information and Systems E84-D, 751–754 (2001).
  16. TriD-Technical Report for 3D Human Modeling Animation Application, ver. 1.0, Opto-Electronics Systems Laboratories, Industrial Technology Research Institute, Taiwan, 2000.
  17. P. Burt, B. Julesz, “Modifications of the classical notion of Panum’s fusional area,” Perception 9, 671–682 (1980).
    [CrossRef]
  18. S. B. Pollard, J. E. W. Mayhew, J. P. Frisby, “PMF: A stereo correspondence algorithm using disparity gradient limit,” Perception 14, 449–470 (1985).
    [CrossRef]
  19. C. Y. Kang, Y. S. Chen, W. H. Hsu, “Automatic approach to mapping a lifelike 2.5D human face,” Image Vision Comput. 12, 5–14 (1994).
    [CrossRef]

2001 (2)

C. Rocchini, P. Cignoni, C. Montani, P. Pingi, R. Scopigno, “A low cost 3D scanner based on structured light,” in Computer Graphics Forum, Proc. EUROGRAPHICS 20, 299–308 (2001).

Y. S. Chen, M. H. Wang, “An approach to perceiving the multi-meaningful-dotted-pattern in a CBP image,” IEICE Trans. Information and Systems E84-D, 751–754 (2001).

2000 (1)

J. P. Siebert, S. J. Marshall, “Human body 3D imaging by speckle texture projection photogrammetry,” Sensor Review 20, 218–226 (2000).
[CrossRef]

1998 (2)

J. Batlle, E. Mouaddib, J. Salvi, “Recent progress in coded structured light as a technique to solve the correspondence problem: a survey,” Pattern Recogn. 31, 963–982 (1998).
[CrossRef]

J. Salvi, J. Batlle, E. Mouaddib, “A robust-coded pattern projection for dynamic 3D scene measurement,” Pattern Recogn. Lett. 19, 1055–1065 (1998).
[CrossRef]

1995 (1)

Y. S. Chen, Y. C. Hsu, “Computer vision on a colour blindness plate,” Image Vision Comput. 13, 463–478 (1995).
[CrossRef]

1994 (2)

C. Y. Kang, Y. S. Chen, W. H. Hsu, “Automatic approach to mapping a lifelike 2.5D human face,” Image Vision Comput. 12, 5–14 (1994).
[CrossRef]

Y. S. Chen, Y. C. Hsu, “Image segmentation of a color-blindness plate,” Appl. Opt. 33, 6818–6822 (1994).
[CrossRef] [PubMed]

1985 (1)

S. B. Pollard, J. E. W. Mayhew, J. P. Frisby, “PMF: A stereo correspondence algorithm using disparity gradient limit,” Perception 14, 449–470 (1985).
[CrossRef]

1984 (1)

H. K. Nishihara, “Practical real-time imaging stereo matcher,” Optical Eng. 23, 536–545 (1984).
[CrossRef]

1980 (1)

P. Burt, B. Julesz, “Modifications of the classical notion of Panum’s fusional area,” Perception 9, 671–682 (1980).
[CrossRef]

1957 (1)

H. T. E. Hertzberg, C. W. Dupertuis, “Stereophotogrammetry as an anthropometric tool,” Photogrammetric Engineering 32, 942–947 (1957).

Batlle, J.

J. Batlle, E. Mouaddib, J. Salvi, “Recent progress in coded structured light as a technique to solve the correspondence problem: a survey,” Pattern Recogn. 31, 963–982 (1998).
[CrossRef]

J. Salvi, J. Batlle, E. Mouaddib, “A robust-coded pattern projection for dynamic 3D scene measurement,” Pattern Recogn. Lett. 19, 1055–1065 (1998).
[CrossRef]

Burt, P.

P. Burt, B. Julesz, “Modifications of the classical notion of Panum’s fusional area,” Perception 9, 671–682 (1980).
[CrossRef]

Chen, Y. S.

Y. S. Chen, M. H. Wang, “An approach to perceiving the multi-meaningful-dotted-pattern in a CBP image,” IEICE Trans. Information and Systems E84-D, 751–754 (2001).

Y. S. Chen, Y. C. Hsu, “Computer vision on a colour blindness plate,” Image Vision Comput. 13, 463–478 (1995).
[CrossRef]

Y. S. Chen, Y. C. Hsu, “Image segmentation of a color-blindness plate,” Appl. Opt. 33, 6818–6822 (1994).
[CrossRef] [PubMed]

C. Y. Kang, Y. S. Chen, W. H. Hsu, “Automatic approach to mapping a lifelike 2.5D human face,” Image Vision Comput. 12, 5–14 (1994).
[CrossRef]

Cignoni, P.

C. Rocchini, P. Cignoni, C. Montani, P. Pingi, R. Scopigno, “A low cost 3D scanner based on structured light,” in Computer Graphics Forum, Proc. EUROGRAPHICS 20, 299–308 (2001).

D’Apuzzo, N.

N. D’Apuzzo, “Automated photogrammetric measurement of human faces,” International Archives of Photogrammetry and Remote Sensing32, Hakodate, Japan, 402–407 (1998).

N. D’Apuzzo, “Human face modeling from multi images,” Proc. 3rd International Image Sensing Seminar on New Development in Digital Photogrammetry, Gifu, Japan, 28–29 (2001).

N. D’Apuzzo, “Modeling human faces with multi-image photogrammetry,” in Three-Dimensional Image Capture and Applications V, Proc. SPIE4661, San Jose, Calif., 191–197 (2002).

Dupertuis, C. W.

H. T. E. Hertzberg, C. W. Dupertuis, “Stereophotogrammetry as an anthropometric tool,” Photogrammetric Engineering 32, 942–947 (1957).

Fraser, C. S.

M. Zhou, C. S. Fraser, “Automated surface extraction in real time photogrammetry,” in XIXth ISPRS Congress, Proc. IAPRS XXXIII, Amsterdam, Netherlands, 943–950 (2000).

Frisby, J. P.

S. B. Pollard, J. E. W. Mayhew, J. P. Frisby, “PMF: A stereo correspondence algorithm using disparity gradient limit,” Perception 14, 449–470 (1985).
[CrossRef]

Grün, A.

A. Grün, “Adaptive least squares correlation: a powerful image matching technique,” South African Journal of Photogrammetry, Remote Sensing and Cartography14, 175–187 (1985).

Gühring, J.

J. Gühring, “Dense 3-D surface acquisition by structured light using off-the-shelf components,” in Photonics West 2001: Videometrics VII, Proc. SPIE4309, San Jose, Calif.220–231 (2001).

Hertzberg, H. T. E.

H. T. E. Hertzberg, C. W. Dupertuis, “Stereophotogrammetry as an anthropometric tool,” Photogrammetric Engineering 32, 942–947 (1957).

Hsu, W. H.

C. Y. Kang, Y. S. Chen, W. H. Hsu, “Automatic approach to mapping a lifelike 2.5D human face,” Image Vision Comput. 12, 5–14 (1994).
[CrossRef]

Hsu, Y. C.

Y. S. Chen, Y. C. Hsu, “Computer vision on a colour blindness plate,” Image Vision Comput. 13, 463–478 (1995).
[CrossRef]

Y. S. Chen, Y. C. Hsu, “Image segmentation of a color-blindness plate,” Appl. Opt. 33, 6818–6822 (1994).
[CrossRef] [PubMed]

Julesz, B.

P. Burt, B. Julesz, “Modifications of the classical notion of Panum’s fusional area,” Perception 9, 671–682 (1980).
[CrossRef]

Kang, C. Y.

C. Y. Kang, Y. S. Chen, W. H. Hsu, “Automatic approach to mapping a lifelike 2.5D human face,” Image Vision Comput. 12, 5–14 (1994).
[CrossRef]

Marshall, S. J.

J. P. Siebert, S. J. Marshall, “Human body 3D imaging by speckle texture projection photogrammetry,” Sensor Review 20, 218–226 (2000).
[CrossRef]

Mayhew, J. E. W.

S. B. Pollard, J. E. W. Mayhew, J. P. Frisby, “PMF: A stereo correspondence algorithm using disparity gradient limit,” Perception 14, 449–470 (1985).
[CrossRef]

Montani, C.

C. Rocchini, P. Cignoni, C. Montani, P. Pingi, R. Scopigno, “A low cost 3D scanner based on structured light,” in Computer Graphics Forum, Proc. EUROGRAPHICS 20, 299–308 (2001).

Mouaddib, E.

J. Salvi, J. Batlle, E. Mouaddib, “A robust-coded pattern projection for dynamic 3D scene measurement,” Pattern Recogn. Lett. 19, 1055–1065 (1998).
[CrossRef]

J. Batlle, E. Mouaddib, J. Salvi, “Recent progress in coded structured light as a technique to solve the correspondence problem: a survey,” Pattern Recogn. 31, 963–982 (1998).
[CrossRef]

Nishihara, H. K.

H. K. Nishihara, “Practical real-time imaging stereo matcher,” Optical Eng. 23, 536–545 (1984).
[CrossRef]

Pingi, P.

C. Rocchini, P. Cignoni, C. Montani, P. Pingi, R. Scopigno, “A low cost 3D scanner based on structured light,” in Computer Graphics Forum, Proc. EUROGRAPHICS 20, 299–308 (2001).

Pollard, S. B.

S. B. Pollard, J. E. W. Mayhew, J. P. Frisby, “PMF: A stereo correspondence algorithm using disparity gradient limit,” Perception 14, 449–470 (1985).
[CrossRef]

Rocchini, C.

C. Rocchini, P. Cignoni, C. Montani, P. Pingi, R. Scopigno, “A low cost 3D scanner based on structured light,” in Computer Graphics Forum, Proc. EUROGRAPHICS 20, 299–308 (2001).

Salvi, J.

J. Batlle, E. Mouaddib, J. Salvi, “Recent progress in coded structured light as a technique to solve the correspondence problem: a survey,” Pattern Recogn. 31, 963–982 (1998).
[CrossRef]

J. Salvi, J. Batlle, E. Mouaddib, “A robust-coded pattern projection for dynamic 3D scene measurement,” Pattern Recogn. Lett. 19, 1055–1065 (1998).
[CrossRef]

Scopigno, R.

C. Rocchini, P. Cignoni, C. Montani, P. Pingi, R. Scopigno, “A low cost 3D scanner based on structured light,” in Computer Graphics Forum, Proc. EUROGRAPHICS 20, 299–308 (2001).

Siebert, J. P.

J. P. Siebert, S. J. Marshall, “Human body 3D imaging by speckle texture projection photogrammetry,” Sensor Review 20, 218–226 (2000).
[CrossRef]

Wang, M. H.

Y. S. Chen, M. H. Wang, “An approach to perceiving the multi-meaningful-dotted-pattern in a CBP image,” IEICE Trans. Information and Systems E84-D, 751–754 (2001).

Zhou, M.

M. Zhou, C. S. Fraser, “Automated surface extraction in real time photogrammetry,” in XIXth ISPRS Congress, Proc. IAPRS XXXIII, Amsterdam, Netherlands, 943–950 (2000).

Appl. Opt. (1)

Computer Graphics Forum, Proc. EUROGRAPHICS (1)

C. Rocchini, P. Cignoni, C. Montani, P. Pingi, R. Scopigno, “A low cost 3D scanner based on structured light,” in Computer Graphics Forum, Proc. EUROGRAPHICS 20, 299–308 (2001).

IEICE Trans. Information and Systems (1)

Y. S. Chen, M. H. Wang, “An approach to perceiving the multi-meaningful-dotted-pattern in a CBP image,” IEICE Trans. Information and Systems E84-D, 751–754 (2001).

Image Vision Comput. (2)

Y. S. Chen, Y. C. Hsu, “Computer vision on a colour blindness plate,” Image Vision Comput. 13, 463–478 (1995).
[CrossRef]

C. Y. Kang, Y. S. Chen, W. H. Hsu, “Automatic approach to mapping a lifelike 2.5D human face,” Image Vision Comput. 12, 5–14 (1994).
[CrossRef]

Optical Eng. (1)

H. K. Nishihara, “Practical real-time imaging stereo matcher,” Optical Eng. 23, 536–545 (1984).
[CrossRef]

Pattern Recogn. (1)

J. Batlle, E. Mouaddib, J. Salvi, “Recent progress in coded structured light as a technique to solve the correspondence problem: a survey,” Pattern Recogn. 31, 963–982 (1998).
[CrossRef]

Pattern Recogn. Lett. (1)

J. Salvi, J. Batlle, E. Mouaddib, “A robust-coded pattern projection for dynamic 3D scene measurement,” Pattern Recogn. Lett. 19, 1055–1065 (1998).
[CrossRef]

Perception (2)

P. Burt, B. Julesz, “Modifications of the classical notion of Panum’s fusional area,” Perception 9, 671–682 (1980).
[CrossRef]

S. B. Pollard, J. E. W. Mayhew, J. P. Frisby, “PMF: A stereo correspondence algorithm using disparity gradient limit,” Perception 14, 449–470 (1985).
[CrossRef]

Photogrammetric Engineering (1)

H. T. E. Hertzberg, C. W. Dupertuis, “Stereophotogrammetry as an anthropometric tool,” Photogrammetric Engineering 32, 942–947 (1957).

Sensor Review (1)

J. P. Siebert, S. J. Marshall, “Human body 3D imaging by speckle texture projection photogrammetry,” Sensor Review 20, 218–226 (2000).
[CrossRef]

Other (7)

N. D’Apuzzo, “Automated photogrammetric measurement of human faces,” International Archives of Photogrammetry and Remote Sensing32, Hakodate, Japan, 402–407 (1998).

N. D’Apuzzo, “Human face modeling from multi images,” Proc. 3rd International Image Sensing Seminar on New Development in Digital Photogrammetry, Gifu, Japan, 28–29 (2001).

N. D’Apuzzo, “Modeling human faces with multi-image photogrammetry,” in Three-Dimensional Image Capture and Applications V, Proc. SPIE4661, San Jose, Calif., 191–197 (2002).

M. Zhou, C. S. Fraser, “Automated surface extraction in real time photogrammetry,” in XIXth ISPRS Congress, Proc. IAPRS XXXIII, Amsterdam, Netherlands, 943–950 (2000).

J. Gühring, “Dense 3-D surface acquisition by structured light using off-the-shelf components,” in Photonics West 2001: Videometrics VII, Proc. SPIE4309, San Jose, Calif.220–231 (2001).

TriD-Technical Report for 3D Human Modeling Animation Application, ver. 1.0, Opto-Electronics Systems Laboratories, Industrial Technology Research Institute, Taiwan, 2000.

A. Grün, “Adaptive least squares correlation: a powerful image matching technique,” South African Journal of Photogrammetry, Remote Sensing and Cartography14, 175–187 (1985).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (16)

Fig. 1
Fig. 1

(a) Our 3D measurement system, (b) The used random speckle pattern, which is sent from the computer and projected via the video projector.

Fig. 2
Fig. 2

(a) Left captured image, (b) right captured image for object 1, (c) left captured image, (d) right captured image for object 2, (e) left captured image, (f) right captured image for object 3.

Fig. 3
Fig. 3

(a), (b) Show the captured left image and right image, respectively. Block image size be 16 × 16, and percentile p = 50% used in Eq. (1), (c) shows the left gray block image and its binary image, where (x 0, y 0) = (150, 150). Based on our distance computation scheme, the best-matching right block image shown in (d) is found with (u f , v f ) = (144, 148). (e) Shows the found correspondence (144, 148) has the minimum value (0.242188) in the distance distribution of a partial searching range.

Fig. 4
Fig. 4

(a) Elementary stereo geometry in canonical configuration. The synthetic stereo pairs, (a) (h = 35, f = 5, z = 6, and disparity = 58.333374), (b) (h = 35, f = 5, z = 14, and disparity = 25.0), for a planar surface with the adopted random speckle pattern shown in Fig. 1(b).

Fig. 5
Fig. 5

(a), (b) Show the 50% white noise generated by a random number generator in C language and are involved in the stereo pairs of Figs. 4(b) and 4(c), respectively.

Fig. 6
Fig. 6

(a) Three cases of planar surfaces with different distances (z = 6, 10, 14) as well as different noise added (noise ratio = 0 ∼ 100%) were analyzed by RMSE. (b)–(d) display the RMSE plots with different cases. All these plots demonstrate that the RMSE of using subpixel compensation will be lower than that without using subpixel compensation support in our approach.

Fig. 7
Fig. 7

(a) Configuration of a sloping surface for measuring ESR and disparity gradients, (b) synthetic stereo pairs of a sloping surface with the range of disparity fields, [17.6, 33.1], (c) plot of measuring the RMSEs with different search ranges, (d) plot of measuring the disparity gradients along the horizontal axis.

Fig. 8
Fig. 8

Surface reconstruction for each case given in Fig. 7.

Fig. 9
Fig. 9

Some reconstructed 3D surfaces with different p values for object 1 by fixing s = 16, without subpixel compensation.

Fig. 10
Fig. 10

Some reconstructed 3D surfaces with different p values for object 1 by fixing s = 16, with 3 × 3 support for subpixel compensation.

Fig. 11
Fig. 11

Some correspondence vector maps for object 1, (a) s = 16, without subpixel compensation, (b) s = 16, with 3 × 3 support for subpixel compensation, (c) s = 8, with 3 × 3 support for subpixel compensation, (d) s = 4, with 5 × 5 support for subpixel compensation.

Fig. 12
Fig. 12

Some reconstructed 3D surfaces with different p values for object 1 by fixing s = 8, where the left-hand side is without subpixel compensation and the right-hand side is with 3 × 3 support for subpixel compensation.

Fig. 13
Fig. 13

Some reconstructed 3D surfaces with different p values for object 1 by fixing s = 4, where the left-hand side is without subpixel compensation and the right-hand side is with 5 × 5 support for subpixel compensation.

Fig. 14
Fig. 14

Some reconstructed 3D surfaces with different p values for object 2 (left-hand side) and object 3 (right-hand side) by fixing s = 4, with 5 × 5 support for subpixel compensation.

Fig. 15
Fig. 15

Viewable results of the reconstructed 3D surfaces for the three objects, with p = 65%, s = 4, as well as 5 × 5 support for subpixel compensation.

Fig. 16
Fig. 16

(a) Texture image of the second author of this paper, (b)–(d) photorealistic visualization by some manipulations on the TriD system.

Tables (1)

Tables Icon

Table 1 Execution Time with Visual C++5.0 on a Pentium II 450 MHz PC

Equations (11)

Equations on this page are rendered with MathJax. Learn more.

THp=zp|cdfzppm2.
bx, y=1if gx, yTHp,0otherwise.
dbx0,y0lx, y; bu0,v0ru, v=x-u2+y-v21/2
dbx0,y0lx, y; bu0,v0r=minbu0,v0ru, v0 dbx0,y0lx, y; bu0,v0ru, v.
dbx0,y0l, bu0,v0r=bx0,y0lx,y0 dbx0,y0lx, y; bu0,v0r.
uf, vf=u0, v0|minu0x0-Rx,x0+Rxv0y0-Ry,y0+Ry dbx0,y0l, bu0,v0r.
ufs=1Nuf,vfA uf,vfs=1Nuf,vfA vf.
z=2hfPr-Pl.
Pl=-h+xfz, Pr=h-xfz.
ESR=maxidisparityi+δ,
disparity gradient=2|Xr-Xl||Xr+Xl|.

Metrics