Abstract

A multimodal cost function, based on the local multimodal image descriptors that combine local structure features (orientation, coherency) and modulation/localization information (amplitude, phase, and spatial frequency) of the monogenic wavelet transform, is proposed to estimate the time-varying disparity maps in the space-time stereo framework. The proposed cost function makes use of a constraint of local orientation, phase, and amplitude congruencies with the weighted coefficients, which are adapted to local image features and are insusceptible to level shift, scaling, and rotation and lighting invariance. Experiments on the synthetic and natural stereo sequences show the estimated results are more robust than the intensity-based sum of standard sum of squared difference cost function.

© 2010 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. Davis, R. Ramamoorthi, and S. Rusinkiewicz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 27, pp. 359-366.
  2. L. Zhang, B. Curless, and S. M. Seitz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 2, pp. 367-374.
  3. M. Magnor and B. Goldlücke, in Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'04) (IEEE, 2004), pp. 365-372.
    [CrossRef]
  4. B. Goldluecke and M. Magnor, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), Vol. 1, pp. 350-355.
    [CrossRef]
  5. Z. M. Liang, H. M. Gao, Z. J. Wang, X. H. Yu, and L. Wu, in Robotic Welding, Intelligence and Automation, T.-J.Tarn, S.-B.Chen, C.Zhou, eds. (Springer, 2007), Vol. 362, pp. 211-218.
    [CrossRef]
  6. B. Barrois and C. Wöhler, in Lecture Notes in Computer Science (Springer, 2008), pp. 507-517.
    [CrossRef]
  7. L. Zhimin, G. Hongming, Z. Guangjun, and L. Wu, Optical Technique 33, 875 (2007).
  8. D. Ville, T. Blu, B. Forster, and M. Unser, in IEEE International Conference on Image Processing (IEEE, 2004), Vol. 1-5, pp. 661-664.
  9. S. C. Olhede and G. Metikas, IEEE Trans. Signal Process. 57, 3426 (2009).
    [CrossRef]
  10. S. Held and M. Storath, Proc. SPIE 7446, 74460Z-1 (2009).
  11. M. Unser, D. Sage, and D. Ville, IEEE Trans. Image Process. 18, 2402 (2009).
    [CrossRef]
  12. N. Kruger and M. Felsberg, Pattern Recogn. Lett. 25, 849 (2004).
    [CrossRef]
  13. J. Li, H. Zhao, X. Zhou, and C. Shi, Opt. Lett. 34, 3514 (2009).
    [CrossRef] [PubMed]
  14. M. Felsberg and G. Sommer, IEEE Trans. Signal Process. 49, 3136 (2001).
    [CrossRef]
  15. Y. Zhang, M. Gong, and Y.-H. Yang, in 5th Canadian Conference on Computer and Robot Vision (2008), pp. 113-120.
    [CrossRef]
  16. T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, in Proceedings of the 23rd International Conference on Image and Vision Computing (2008).
  17. R. Reulke, A. Luber, M. Haberjahn, and B. Piltz, in Proceedings of the 3D-NordOst (2009).

2009 (5)

S. C. Olhede and G. Metikas, IEEE Trans. Signal Process. 57, 3426 (2009).
[CrossRef]

S. Held and M. Storath, Proc. SPIE 7446, 74460Z-1 (2009).

M. Unser, D. Sage, and D. Ville, IEEE Trans. Image Process. 18, 2402 (2009).
[CrossRef]

R. Reulke, A. Luber, M. Haberjahn, and B. Piltz, in Proceedings of the 3D-NordOst (2009).

J. Li, H. Zhao, X. Zhou, and C. Shi, Opt. Lett. 34, 3514 (2009).
[CrossRef] [PubMed]

2008 (3)

B. Barrois and C. Wöhler, in Lecture Notes in Computer Science (Springer, 2008), pp. 507-517.
[CrossRef]

Y. Zhang, M. Gong, and Y.-H. Yang, in 5th Canadian Conference on Computer and Robot Vision (2008), pp. 113-120.
[CrossRef]

T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, in Proceedings of the 23rd International Conference on Image and Vision Computing (2008).

2007 (2)

L. Zhimin, G. Hongming, Z. Guangjun, and L. Wu, Optical Technique 33, 875 (2007).

Z. M. Liang, H. M. Gao, Z. J. Wang, X. H. Yu, and L. Wu, in Robotic Welding, Intelligence and Automation, T.-J.Tarn, S.-B.Chen, C.Zhou, eds. (Springer, 2007), Vol. 362, pp. 211-218.
[CrossRef]

2004 (4)

D. Ville, T. Blu, B. Forster, and M. Unser, in IEEE International Conference on Image Processing (IEEE, 2004), Vol. 1-5, pp. 661-664.

N. Kruger and M. Felsberg, Pattern Recogn. Lett. 25, 849 (2004).
[CrossRef]

M. Magnor and B. Goldlücke, in Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'04) (IEEE, 2004), pp. 365-372.
[CrossRef]

B. Goldluecke and M. Magnor, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), Vol. 1, pp. 350-355.
[CrossRef]

2003 (2)

J. Davis, R. Ramamoorthi, and S. Rusinkiewicz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 27, pp. 359-366.

L. Zhang, B. Curless, and S. M. Seitz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 2, pp. 367-374.

2001 (1)

M. Felsberg and G. Sommer, IEEE Trans. Signal Process. 49, 3136 (2001).
[CrossRef]

Barrois, B.

B. Barrois and C. Wöhler, in Lecture Notes in Computer Science (Springer, 2008), pp. 507-517.
[CrossRef]

Blu, T.

D. Ville, T. Blu, B. Forster, and M. Unser, in IEEE International Conference on Image Processing (IEEE, 2004), Vol. 1-5, pp. 661-664.

Curless, B.

L. Zhang, B. Curless, and S. M. Seitz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 2, pp. 367-374.

Davis, J.

J. Davis, R. Ramamoorthi, and S. Rusinkiewicz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 27, pp. 359-366.

Felsberg, M.

N. Kruger and M. Felsberg, Pattern Recogn. Lett. 25, 849 (2004).
[CrossRef]

M. Felsberg and G. Sommer, IEEE Trans. Signal Process. 49, 3136 (2001).
[CrossRef]

Forster, B.

D. Ville, T. Blu, B. Forster, and M. Unser, in IEEE International Conference on Image Processing (IEEE, 2004), Vol. 1-5, pp. 661-664.

Gao, H. M.

Z. M. Liang, H. M. Gao, Z. J. Wang, X. H. Yu, and L. Wu, in Robotic Welding, Intelligence and Automation, T.-J.Tarn, S.-B.Chen, C.Zhou, eds. (Springer, 2007), Vol. 362, pp. 211-218.
[CrossRef]

Goldlücke, B.

M. Magnor and B. Goldlücke, in Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'04) (IEEE, 2004), pp. 365-372.
[CrossRef]

Goldluecke, B.

B. Goldluecke and M. Magnor, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), Vol. 1, pp. 350-355.
[CrossRef]

Gong, M.

Y. Zhang, M. Gong, and Y.-H. Yang, in 5th Canadian Conference on Computer and Robot Vision (2008), pp. 113-120.
[CrossRef]

Guangjun, Z.

L. Zhimin, G. Hongming, Z. Guangjun, and L. Wu, Optical Technique 33, 875 (2007).

Haberjahn, M.

R. Reulke, A. Luber, M. Haberjahn, and B. Piltz, in Proceedings of the 3D-NordOst (2009).

Held, S.

S. Held and M. Storath, Proc. SPIE 7446, 74460Z-1 (2009).

Hongming, G.

L. Zhimin, G. Hongming, Z. Guangjun, and L. Wu, Optical Technique 33, 875 (2007).

Klette, R.

T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, in Proceedings of the 23rd International Conference on Image and Vision Computing (2008).

Kruger, N.

N. Kruger and M. Felsberg, Pattern Recogn. Lett. 25, 849 (2004).
[CrossRef]

Li, J.

Liang, Z. M.

Z. M. Liang, H. M. Gao, Z. J. Wang, X. H. Yu, and L. Wu, in Robotic Welding, Intelligence and Automation, T.-J.Tarn, S.-B.Chen, C.Zhou, eds. (Springer, 2007), Vol. 362, pp. 211-218.
[CrossRef]

Luber, A.

R. Reulke, A. Luber, M. Haberjahn, and B. Piltz, in Proceedings of the 3D-NordOst (2009).

Magnor, M.

M. Magnor and B. Goldlücke, in Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'04) (IEEE, 2004), pp. 365-372.
[CrossRef]

B. Goldluecke and M. Magnor, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), Vol. 1, pp. 350-355.
[CrossRef]

Metikas, G.

S. C. Olhede and G. Metikas, IEEE Trans. Signal Process. 57, 3426 (2009).
[CrossRef]

Milburn, J.

T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, in Proceedings of the 23rd International Conference on Image and Vision Computing (2008).

Olhede, S. C.

S. C. Olhede and G. Metikas, IEEE Trans. Signal Process. 57, 3426 (2009).
[CrossRef]

Piltz, B.

R. Reulke, A. Luber, M. Haberjahn, and B. Piltz, in Proceedings of the 3D-NordOst (2009).

Rabe, C.

T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, in Proceedings of the 23rd International Conference on Image and Vision Computing (2008).

Ramamoorthi, R.

J. Davis, R. Ramamoorthi, and S. Rusinkiewicz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 27, pp. 359-366.

Reulke, R.

R. Reulke, A. Luber, M. Haberjahn, and B. Piltz, in Proceedings of the 3D-NordOst (2009).

Rusinkiewicz, S.

J. Davis, R. Ramamoorthi, and S. Rusinkiewicz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 27, pp. 359-366.

Sage, D.

M. Unser, D. Sage, and D. Ville, IEEE Trans. Image Process. 18, 2402 (2009).
[CrossRef]

Seitz, S. M.

L. Zhang, B. Curless, and S. M. Seitz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 2, pp. 367-374.

Shi, C.

Sommer, G.

M. Felsberg and G. Sommer, IEEE Trans. Signal Process. 49, 3136 (2001).
[CrossRef]

Storath, M.

S. Held and M. Storath, Proc. SPIE 7446, 74460Z-1 (2009).

Unser, M.

M. Unser, D. Sage, and D. Ville, IEEE Trans. Image Process. 18, 2402 (2009).
[CrossRef]

D. Ville, T. Blu, B. Forster, and M. Unser, in IEEE International Conference on Image Processing (IEEE, 2004), Vol. 1-5, pp. 661-664.

Vaudrey, T.

T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, in Proceedings of the 23rd International Conference on Image and Vision Computing (2008).

Ville, D.

M. Unser, D. Sage, and D. Ville, IEEE Trans. Image Process. 18, 2402 (2009).
[CrossRef]

D. Ville, T. Blu, B. Forster, and M. Unser, in IEEE International Conference on Image Processing (IEEE, 2004), Vol. 1-5, pp. 661-664.

Wang, Z. J.

Z. M. Liang, H. M. Gao, Z. J. Wang, X. H. Yu, and L. Wu, in Robotic Welding, Intelligence and Automation, T.-J.Tarn, S.-B.Chen, C.Zhou, eds. (Springer, 2007), Vol. 362, pp. 211-218.
[CrossRef]

Wöhler, C.

B. Barrois and C. Wöhler, in Lecture Notes in Computer Science (Springer, 2008), pp. 507-517.
[CrossRef]

Wu, L.

Z. M. Liang, H. M. Gao, Z. J. Wang, X. H. Yu, and L. Wu, in Robotic Welding, Intelligence and Automation, T.-J.Tarn, S.-B.Chen, C.Zhou, eds. (Springer, 2007), Vol. 362, pp. 211-218.
[CrossRef]

L. Zhimin, G. Hongming, Z. Guangjun, and L. Wu, Optical Technique 33, 875 (2007).

Yang, Y.-H.

Y. Zhang, M. Gong, and Y.-H. Yang, in 5th Canadian Conference on Computer and Robot Vision (2008), pp. 113-120.
[CrossRef]

Yu, X. H.

Z. M. Liang, H. M. Gao, Z. J. Wang, X. H. Yu, and L. Wu, in Robotic Welding, Intelligence and Automation, T.-J.Tarn, S.-B.Chen, C.Zhou, eds. (Springer, 2007), Vol. 362, pp. 211-218.
[CrossRef]

Zhang, L.

L. Zhang, B. Curless, and S. M. Seitz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 2, pp. 367-374.

Zhang, Y.

Y. Zhang, M. Gong, and Y.-H. Yang, in 5th Canadian Conference on Computer and Robot Vision (2008), pp. 113-120.
[CrossRef]

Zhao, H.

Zhimin, L.

L. Zhimin, G. Hongming, Z. Guangjun, and L. Wu, Optical Technique 33, 875 (2007).

Zhou, X.

IEEE Trans. Image Process. (1)

M. Unser, D. Sage, and D. Ville, IEEE Trans. Image Process. 18, 2402 (2009).
[CrossRef]

IEEE Trans. Signal Process. (2)

M. Felsberg and G. Sommer, IEEE Trans. Signal Process. 49, 3136 (2001).
[CrossRef]

S. C. Olhede and G. Metikas, IEEE Trans. Signal Process. 57, 3426 (2009).
[CrossRef]

Opt. Lett. (1)

Optical Technique (1)

L. Zhimin, G. Hongming, Z. Guangjun, and L. Wu, Optical Technique 33, 875 (2007).

Pattern Recogn. Lett. (1)

N. Kruger and M. Felsberg, Pattern Recogn. Lett. 25, 849 (2004).
[CrossRef]

Proc. SPIE (1)

S. Held and M. Storath, Proc. SPIE 7446, 74460Z-1 (2009).

Other (10)

D. Ville, T. Blu, B. Forster, and M. Unser, in IEEE International Conference on Image Processing (IEEE, 2004), Vol. 1-5, pp. 661-664.

J. Davis, R. Ramamoorthi, and S. Rusinkiewicz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 27, pp. 359-366.

L. Zhang, B. Curless, and S. M. Seitz, in Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2003), Vol. 2, pp. 367-374.

M. Magnor and B. Goldlücke, in Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'04) (IEEE, 2004), pp. 365-372.
[CrossRef]

B. Goldluecke and M. Magnor, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), Vol. 1, pp. 350-355.
[CrossRef]

Z. M. Liang, H. M. Gao, Z. J. Wang, X. H. Yu, and L. Wu, in Robotic Welding, Intelligence and Automation, T.-J.Tarn, S.-B.Chen, C.Zhou, eds. (Springer, 2007), Vol. 362, pp. 211-218.
[CrossRef]

B. Barrois and C. Wöhler, in Lecture Notes in Computer Science (Springer, 2008), pp. 507-517.
[CrossRef]

Y. Zhang, M. Gong, and Y.-H. Yang, in 5th Canadian Conference on Computer and Robot Vision (2008), pp. 113-120.
[CrossRef]

T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, in Proceedings of the 23rd International Conference on Image and Vision Computing (2008).

R. Reulke, A. Luber, M. Haberjahn, and B. Piltz, in Proceedings of the 3D-NordOst (2009).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1
Fig. 1

Dense disparity maps for synthetic stereo sequences: left image (left) and MWT-based (middle) and SSSD-based (right) disparity maps at frame 10 (top), 30 (middle), and 50 (bottom), respectively.

Fig. 2
Fig. 2

Statistic results of RMSE and PBD for synthetic stereo sequences: (a) RMSE, (b) PBD. MWT, red crosses; SSSD, green asterisks.

Fig. 3
Fig. 3

Feature points matching map for 45th frame of natural stereo sequences: (a) left and (b) right images, (c) putative and (d) inlying match points for MWT, (e) putative and (f) inlying match points for SSSD. MWT, red crosses; SSSD, green asterisks.

Fig. 4
Fig. 4

Experiment results for natural stereo sequences: (a) number of putative matches; (b), (c) number and percentage of inlying matches. MWT, red crosses; SSSD, green asterisks.

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

E ( d ( x , y , t ) ) = t T 0 ( x , y ) W 0 e ( I l ( x , y , t ) , I r ( x d ̃ 0 , y , t ) ) ,
ψ ( x ) = ( Δ ) γ 2 φ 2 γ ( D x ) , ψ ( x ) = R s * ψ ( x ) ,
e ( ε l , ε r ) = χ ( Δ θ ) 2 + ( 1 χ ) ( δ ( Δ ϕ ) 2 + ( 1 δ ) ( Δ A ) 2 ) ,

Metrics