Abstract

Stereo matching, a technique for acquiring depth information from many planar images obtained by several cameras, was developed several decades ago. Recently a novel depth-extraction technique that uses a lens array instead of several cameras has attracted much attention because of the advantages offered by its compact system configuration. We present a novel depth-extraction method that uses a lens array consisting of vertically long rectangular lens elements. The proposed method rearranges the horizontal positions of the pixels from the collection of perspective images while it leaves the vertical positions of the pixels unchanged. To these rearranged images we apply a correlation-based multibaseline stereo algorithm in properly modified form. The main feature of the proposed method is the inverse dependency of the disparity in depth between horizontal and vertical directions. This inverse dependency permits the extraction of exact depth from extremely periodically patterned object scenes and reduces quantization error in the depth extraction.

© 2004 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. D. Marr, T. Poggio, “A computational theory of human stereo vision,” Proc. R. Soc. London Ser. B 204, 301–328 (1979).
    [CrossRef]
  2. S. T. Barnard, M. A. Fischler, “Stereo vision,” in Encyclopedia of Artificial Intelligence, S. C. Shapiro, ed. (Wiley, New York, 1987), pp. 1083–1090.
  3. M. Yachida, Y. Kitamura, M. Kimachi, “Trinocular vision: new approach for correspondence problem,” presented at the International Conference on Pattern Recognition, Paris, 28–31 October, 1986.
  4. J.-H. Park, S.-W. Min, S. Jung, B. Lee, “A new stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
    [CrossRef]
  5. C. Wu, A. Aggoun, M. McCormick, S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, A. J. Woods, J. O. Merritt, S. A. Benton, M. T. Bolas, eds., Proc. SPIE4660, 135–145 (2002).
    [CrossRef]
  6. J.-H. Park, S.-W. Min, S. Jung, B. Lee, “Analysis of viewing parameters for two display methods based on integral photography,” Appl. Opt. 40, 5217–5232 (2001).
    [CrossRef]
  7. B. Lee, S. Jung, J.-H. Park, “Viewing-angle-enhanced integral imaging using lens switching,” Opt. Lett. 27, 818–820 (2002).
    [CrossRef]
  8. S. Jung, J.-H. Park, B. Lee, B. Javidi, “Viewing-angle-enhanced integral 3D imaging using double display devices with masks,” Opt. Eng. 41, 2389–2390 (2002).
    [CrossRef]
  9. M. Okutomi, T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15, 353–363 (1993).
    [CrossRef]
  10. H. Adelson, Y. A. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
    [CrossRef]
  11. F. Okano, H. Hoshino, J. Arai, I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997).
    [CrossRef] [PubMed]
  12. C. Chang, S. Chatterjee, “Quantization error analysis in stereo vision,” presented at the Twenty-Sixth Asilomar Conference on Signals, Systems and Computers, Pacific Grove, Calif., 26–28 October 1992.

2002 (2)

S. Jung, J.-H. Park, B. Lee, B. Javidi, “Viewing-angle-enhanced integral 3D imaging using double display devices with masks,” Opt. Eng. 41, 2389–2390 (2002).
[CrossRef]

B. Lee, S. Jung, J.-H. Park, “Viewing-angle-enhanced integral imaging using lens switching,” Opt. Lett. 27, 818–820 (2002).
[CrossRef]

2001 (1)

1997 (1)

1993 (1)

M. Okutomi, T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15, 353–363 (1993).
[CrossRef]

1992 (1)

H. Adelson, Y. A. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

1979 (1)

D. Marr, T. Poggio, “A computational theory of human stereo vision,” Proc. R. Soc. London Ser. B 204, 301–328 (1979).
[CrossRef]

Adelson, H.

H. Adelson, Y. A. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

Aggoun, A.

C. Wu, A. Aggoun, M. McCormick, S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, A. J. Woods, J. O. Merritt, S. A. Benton, M. T. Bolas, eds., Proc. SPIE4660, 135–145 (2002).
[CrossRef]

Arai, J.

Barnard, S. T.

S. T. Barnard, M. A. Fischler, “Stereo vision,” in Encyclopedia of Artificial Intelligence, S. C. Shapiro, ed. (Wiley, New York, 1987), pp. 1083–1090.

Chang, C.

C. Chang, S. Chatterjee, “Quantization error analysis in stereo vision,” presented at the Twenty-Sixth Asilomar Conference on Signals, Systems and Computers, Pacific Grove, Calif., 26–28 October 1992.

Chatterjee, S.

C. Chang, S. Chatterjee, “Quantization error analysis in stereo vision,” presented at the Twenty-Sixth Asilomar Conference on Signals, Systems and Computers, Pacific Grove, Calif., 26–28 October 1992.

Fischler, M. A.

S. T. Barnard, M. A. Fischler, “Stereo vision,” in Encyclopedia of Artificial Intelligence, S. C. Shapiro, ed. (Wiley, New York, 1987), pp. 1083–1090.

Hoshino, H.

Javidi, B.

S. Jung, J.-H. Park, B. Lee, B. Javidi, “Viewing-angle-enhanced integral 3D imaging using double display devices with masks,” Opt. Eng. 41, 2389–2390 (2002).
[CrossRef]

Jung, S.

S. Jung, J.-H. Park, B. Lee, B. Javidi, “Viewing-angle-enhanced integral 3D imaging using double display devices with masks,” Opt. Eng. 41, 2389–2390 (2002).
[CrossRef]

B. Lee, S. Jung, J.-H. Park, “Viewing-angle-enhanced integral imaging using lens switching,” Opt. Lett. 27, 818–820 (2002).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “Analysis of viewing parameters for two display methods based on integral photography,” Appl. Opt. 40, 5217–5232 (2001).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “A new stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

Kanade, T.

M. Okutomi, T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15, 353–363 (1993).
[CrossRef]

Kimachi, M.

M. Yachida, Y. Kitamura, M. Kimachi, “Trinocular vision: new approach for correspondence problem,” presented at the International Conference on Pattern Recognition, Paris, 28–31 October, 1986.

Kitamura, Y.

M. Yachida, Y. Kitamura, M. Kimachi, “Trinocular vision: new approach for correspondence problem,” presented at the International Conference on Pattern Recognition, Paris, 28–31 October, 1986.

Kung, S. Y.

C. Wu, A. Aggoun, M. McCormick, S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, A. J. Woods, J. O. Merritt, S. A. Benton, M. T. Bolas, eds., Proc. SPIE4660, 135–145 (2002).
[CrossRef]

Lee, B.

S. Jung, J.-H. Park, B. Lee, B. Javidi, “Viewing-angle-enhanced integral 3D imaging using double display devices with masks,” Opt. Eng. 41, 2389–2390 (2002).
[CrossRef]

B. Lee, S. Jung, J.-H. Park, “Viewing-angle-enhanced integral imaging using lens switching,” Opt. Lett. 27, 818–820 (2002).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “Analysis of viewing parameters for two display methods based on integral photography,” Appl. Opt. 40, 5217–5232 (2001).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “A new stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

Marr, D.

D. Marr, T. Poggio, “A computational theory of human stereo vision,” Proc. R. Soc. London Ser. B 204, 301–328 (1979).
[CrossRef]

McCormick, M.

C. Wu, A. Aggoun, M. McCormick, S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, A. J. Woods, J. O. Merritt, S. A. Benton, M. T. Bolas, eds., Proc. SPIE4660, 135–145 (2002).
[CrossRef]

Min, S.-W.

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “Analysis of viewing parameters for two display methods based on integral photography,” Appl. Opt. 40, 5217–5232 (2001).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “A new stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

Okano, F.

Okutomi, M.

M. Okutomi, T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15, 353–363 (1993).
[CrossRef]

Park, J.-H.

B. Lee, S. Jung, J.-H. Park, “Viewing-angle-enhanced integral imaging using lens switching,” Opt. Lett. 27, 818–820 (2002).
[CrossRef]

S. Jung, J.-H. Park, B. Lee, B. Javidi, “Viewing-angle-enhanced integral 3D imaging using double display devices with masks,” Opt. Eng. 41, 2389–2390 (2002).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “Analysis of viewing parameters for two display methods based on integral photography,” Appl. Opt. 40, 5217–5232 (2001).
[CrossRef]

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “A new stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

Poggio, T.

D. Marr, T. Poggio, “A computational theory of human stereo vision,” Proc. R. Soc. London Ser. B 204, 301–328 (1979).
[CrossRef]

Wang, Y. A.

H. Adelson, Y. A. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

Wu, C.

C. Wu, A. Aggoun, M. McCormick, S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, A. J. Woods, J. O. Merritt, S. A. Benton, M. T. Bolas, eds., Proc. SPIE4660, 135–145 (2002).
[CrossRef]

Yachida, M.

M. Yachida, Y. Kitamura, M. Kimachi, “Trinocular vision: new approach for correspondence problem,” presented at the International Conference on Pattern Recognition, Paris, 28–31 October, 1986.

Yuyama, I.

Appl. Opt. (2)

IEEE Trans. Pattern Anal. Mach. Intell. (2)

M. Okutomi, T. Kanade, “A multiple-baseline stereo,” IEEE Trans. Pattern Anal. Mach. Intell. 15, 353–363 (1993).
[CrossRef]

H. Adelson, Y. A. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 99–106 (1992).
[CrossRef]

Opt. Eng. (1)

S. Jung, J.-H. Park, B. Lee, B. Javidi, “Viewing-angle-enhanced integral 3D imaging using double display devices with masks,” Opt. Eng. 41, 2389–2390 (2002).
[CrossRef]

Opt. Lett. (1)

Proc. R. Soc. London Ser. B (1)

D. Marr, T. Poggio, “A computational theory of human stereo vision,” Proc. R. Soc. London Ser. B 204, 301–328 (1979).
[CrossRef]

Other (5)

S. T. Barnard, M. A. Fischler, “Stereo vision,” in Encyclopedia of Artificial Intelligence, S. C. Shapiro, ed. (Wiley, New York, 1987), pp. 1083–1090.

M. Yachida, Y. Kitamura, M. Kimachi, “Trinocular vision: new approach for correspondence problem,” presented at the International Conference on Pattern Recognition, Paris, 28–31 October, 1986.

J.-H. Park, S.-W. Min, S. Jung, B. Lee, “A new stereovision scheme using a camera and a lens array,” in Algorithms and Systems for Optical Information Processing V, B. Javidi, D. Psaltis, eds., Proc. SPIE4471, 73–80 (2001).
[CrossRef]

C. Wu, A. Aggoun, M. McCormick, S. Y. Kung, “Depth extraction from unidirectional image using a modified multi-baseline technique,” in Stereoscopic Displays and Virtual Reality Systems IX, A. J. Woods, J. O. Merritt, S. A. Benton, M. T. Bolas, eds., Proc. SPIE4660, 135–145 (2002).
[CrossRef]

C. Chang, S. Chatterjee, “Quantization error analysis in stereo vision,” presented at the Twenty-Sixth Asilomar Conference on Signals, Systems and Computers, Pacific Grove, Calif., 26–28 October 1992.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1
Fig. 1

Conceptual diagram of the proposed method for depth extraction.

Fig. 2
Fig. 2

Procedure for modifying the elemental images.

Fig. 3
Fig. 3

Computer-generated object.

Fig. 4
Fig. 4

Examples of elemental images and modified elemental images: (a) part of the original elemental images, (b) part of the modified elemental images, (c) horizontal and vertical disparity in the modified elemental images.

Fig. 5
Fig. 5

Field of view and sampling rate in the horizontal direction: (a) original elemental image, (b) modified elemental image.

Fig. 6
Fig. 6

Geometry of the proposed method.

Fig. 7
Fig. 7

Illustration of the elimination of ambiguity: (a) vertical SSD functions and their sum, (b) horizontal SSD functions and their sum, (c) our evaluation function, which is the sum of the vertical and the horizontal SSD functions.

Fig. 8
Fig. 8

Depth maps detected by horizontal and vertical SSD functions: (a) object with inclined depth from 400 to 1000 mm, (b) depth map detected by the vertical SSD function, (c) depth map detected by the horizontal SSD function.

Fig. 9
Fig. 9

Experimental setup.

Fig. 10
Fig. 10

Objects (a) with a periodic pattern, for experiment 1 and (b) with an inclined depth, for experiment 2.

Fig. 11
Fig. 11

For object 1: (a) part of the captured elemental images, (b) part of the modified elemental images.

Fig. 12
Fig. 12

Detected disparity maps from (a) the sum of the vertical evaluation functions, (b) the sum of the horizontal evaluation functions, and (c) the sum of both the horizontal and the vertical evaluation functions.

Fig. 13
Fig. 13

Example of the evaluation functions in experiment 1.

Fig. 14
Fig. 14

Detected depth profiles when the object depth is 120–170 mm from the sum of (a) the vertical or (b) the horizontal evaluation functions, when the object depth is 220–270 mm from the sum of (c) the vertical or (d) the horizontal evaluation functions, and when the object depth is 300–350 mm from the sum of (e) the vertical or (f) the horizontal evaluation functions.

Tables (2)

Tables Icon

Table 1 Comparison of Lens Array Methods

Tables Icon

Table 2 Specifications of the Experimental Setup

Equations (19)

Equations on this page are rendered with MathJax. Learn more.

xqx, yqy=fcfafdszoqxφx-xo, fcfafdszoqyφy-yo,
upx, vpy=1φxszom px+xo, mszopyφy-yo,
Δupx1,px2=szoφxmpx2-px1,
Δvpy1,py2=φymszopy2-py1.
fpx,pyu, v=Oφxu-szopxm, φypy-szovm, zo.
e[px1py1,[px2py2u, v, ξ=i,jWfpx1py1u+i, v+j-fpx2py2u+i+ spx2-px1mφxξ,v+j+mφyξpy2-py1s2,
e[px1py1],[px2py2]u, v, ξ=i,jWOφxu+i-spx1mξo, -sv+jmξo+φypy1, 1ξo-Oφxu+i-spx1mξ+spx2m×1ξ-1ξo, -sv+jmξo+φypy1ξξo+φypy2×1-ξξo, 1ξo2,
C[px1,py1]u, v, ξ=px2py2 e[px1,py1],[px2,py2]u, v, ξ.
e[px,py],[px,py2]ξ=i,jWOφxu+i-spxmξo, φypy-sv+jmξo, zo-Oφxu+i-spxmξo, φypyξξo-sv+jmξo+φypy21-ξξo, zo2,
e[px,py],[px2,py]ξ=i,jWOφxu+i-spxmξo, φypy-sv+jmξo, zo-Oφxu+i-spxmξ+spx2m1ξ-1ξo,φypy-sv+jmξo, zo2.
ξ=ξo1+bTφypy2-py,
ξ=ξospx2-pxspx2-px+ξomaT,
E|δzyz=sz23mφy|py1-py2|.
fδΔu=fupx1fupx2=1-|δΔu|, 0|δΔu|1,
dδΔudδzx=s|px1-px2|mφx.
fδzx=fδΔudδΔudδzx=s|px1-px2|mφx1-s|px1-px2|mφx|δzx|0 for0|δzx|mφxs|px1-px2|otherwise
E|δzx|=- |δzx|fδzxdδzx=mφx3s|px1-px2|.
e[px1py1],[px2py2]u, v, ξ=i,jWfpx1py1u+i, v+jEfpx1py1u, v-fpx2py2u+i+spx2-px1mφxξ, v+j+mφyξpy2-py1sEfpx1py1u+spx2-px1mφxξ, v+mφyξpy2-py1s2,
Efpx1py1u, v=i,jWfpx1py1u+i, v+jNW,

Metrics