Abstract

Disparity estimation for binocular images is an important problem for many visual tasks such as 3D environment reconstruction, digital hologram, virtual reality, robot navigation, etc. Conventional approaches are based on brightness constancy assumption to establish spatial correspondences between a pair of images. However, in the presence of large illumination variation and serious noisy contamination, conventional approaches fail to generate accurate disparity maps. To have robust disparity estimation in these situations, we first propose a model - color monogenic curvature phase to describe local features of color images by embedding the monogenic curvature signal into the quaternion representation. Then a multiscale framework to estimate disparities is proposed by coupling the advantages of the color monogenic curvature phase and mutual information. Both indoor and outdoor images with large brightness variation are used in the experiments, and the results demonstrate that our approach can achieve a good performance even in the conditions of large illumination change and serious noisy contamination.

© 2012 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 401–406 (1998).
    [CrossRef]
  2. H. Moravec, “Toward automatic visual obstacle avoidance,” in Proceedings of 5th International Joint Conference on Artificial Intelligence, (Morgan Kaufmann, 1977), pp. 584–590.
  3. C. Fookes, M. Bennamoun, and A. Lamanna, “Improved stereo image matching using mutual information and hierarchical prior probabilities,” in Proceedings of 16th International Conference on Pattern Recognition, (IEEE, 2002), pp. 937–940.
  4. I. Sarkar and M. Bansal, “A wavelet-based multiresolution approach to solve the stereo correspondence problem using mutual information,” IEEE Trans. Syst. Man. Cybern., B: Cybern. 37, 1009–1014 (2007).
    [CrossRef]
  5. A. Geiger, M. Roser, and R. Urtasun, “Efficient large-scale stereo matching,” in Proceedings of 10th Asian conference on Computer vision - Volume Part I, (Springer-Verlag, 2011), pp. 25–38.
  6. A. V. Oppenheim, “The importance of phase in signals,” Proc. IEEE 69, 529–541 (1981).
    [CrossRef]
  7. M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49, 3136–3144 (2001).
    [CrossRef]
  8. M. Felsberg and G. Sommer, “The monogenic scale-space: a unifying approach to phase-based image processing in scale-space,” J. Math. Imaging Vision 21, 5–26 (2004).
    [CrossRef]
  9. G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision 40, 269–284 (2011).
    [CrossRef]
  10. D. Zang and G. Sommer, “Signal modeling for two-dimensional image structures,” J. Visual Commun. Image 18, 81–99 (2007).
    [CrossRef]
  11. D. Zang, J. Li, and D. Zhang, “Robust visual correspondence computation using monogenic curvature phase based mutual information,” Opt. Lett. 37, 10–12 (2012).
    [CrossRef] [PubMed]
  12. F. Brackx, B. D. Knock, and H. D. Schepper, “Generalized multidimensional hilbert transforms in clifford analysis,” Int. J. Math. Math. Sci. 2006, 98145 (2006).
    [CrossRef]
  13. S. J. Sangwine, “Fourier transforms of color images using quaternion or hypercomplex numbers,” Electron. Lett. 32, 1979–1980 (1996).
    [CrossRef]
  14. N. L. Bihan and S. J. Sangwine, “Quaternion principal component analysis of color images,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2003), pp. 809–812.
  15. http://vision.middlebury.edu/stereo/ .
  16. J. Kim, V. Kolmogorov, and R. Zabih, “Visual correspondence using energy minimization and mutual information,” in Proceedings of IEEE International Conference on Computer Vision, (IEEE, 2003), pp. 1033–1040.
  17. Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001).
    [CrossRef]
  18. D. Scharstein and C. Pal, “Learning conditional random fields for stereo,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2007), pp. 1–8.

2012 (1)

2011 (1)

G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision 40, 269–284 (2011).
[CrossRef]

2007 (2)

D. Zang and G. Sommer, “Signal modeling for two-dimensional image structures,” J. Visual Commun. Image 18, 81–99 (2007).
[CrossRef]

I. Sarkar and M. Bansal, “A wavelet-based multiresolution approach to solve the stereo correspondence problem using mutual information,” IEEE Trans. Syst. Man. Cybern., B: Cybern. 37, 1009–1014 (2007).
[CrossRef]

2006 (1)

F. Brackx, B. D. Knock, and H. D. Schepper, “Generalized multidimensional hilbert transforms in clifford analysis,” Int. J. Math. Math. Sci. 2006, 98145 (2006).
[CrossRef]

2004 (1)

M. Felsberg and G. Sommer, “The monogenic scale-space: a unifying approach to phase-based image processing in scale-space,” J. Math. Imaging Vision 21, 5–26 (2004).
[CrossRef]

2001 (2)

Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001).
[CrossRef]

M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49, 3136–3144 (2001).
[CrossRef]

1998 (1)

S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 401–406 (1998).
[CrossRef]

1996 (1)

S. J. Sangwine, “Fourier transforms of color images using quaternion or hypercomplex numbers,” Electron. Lett. 32, 1979–1980 (1996).
[CrossRef]

1981 (1)

A. V. Oppenheim, “The importance of phase in signals,” Proc. IEEE 69, 529–541 (1981).
[CrossRef]

Bansal, M.

I. Sarkar and M. Bansal, “A wavelet-based multiresolution approach to solve the stereo correspondence problem using mutual information,” IEEE Trans. Syst. Man. Cybern., B: Cybern. 37, 1009–1014 (2007).
[CrossRef]

Bennamoun, M.

C. Fookes, M. Bennamoun, and A. Lamanna, “Improved stereo image matching using mutual information and hierarchical prior probabilities,” in Proceedings of 16th International Conference on Pattern Recognition, (IEEE, 2002), pp. 937–940.

Berthier, M.

G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision 40, 269–284 (2011).
[CrossRef]

Bihan, N. L.

N. L. Bihan and S. J. Sangwine, “Quaternion principal component analysis of color images,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2003), pp. 809–812.

Birchfield, S.

S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 401–406 (1998).
[CrossRef]

Boykov, Y.

Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001).
[CrossRef]

Brackx, F.

F. Brackx, B. D. Knock, and H. D. Schepper, “Generalized multidimensional hilbert transforms in clifford analysis,” Int. J. Math. Math. Sci. 2006, 98145 (2006).
[CrossRef]

Courtellemont, P.

G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision 40, 269–284 (2011).
[CrossRef]

Demarcq, G.

G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision 40, 269–284 (2011).
[CrossRef]

Felsberg, M.

M. Felsberg and G. Sommer, “The monogenic scale-space: a unifying approach to phase-based image processing in scale-space,” J. Math. Imaging Vision 21, 5–26 (2004).
[CrossRef]

M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49, 3136–3144 (2001).
[CrossRef]

Fookes, C.

C. Fookes, M. Bennamoun, and A. Lamanna, “Improved stereo image matching using mutual information and hierarchical prior probabilities,” in Proceedings of 16th International Conference on Pattern Recognition, (IEEE, 2002), pp. 937–940.

Geiger, A.

A. Geiger, M. Roser, and R. Urtasun, “Efficient large-scale stereo matching,” in Proceedings of 10th Asian conference on Computer vision - Volume Part I, (Springer-Verlag, 2011), pp. 25–38.

Kim, J.

J. Kim, V. Kolmogorov, and R. Zabih, “Visual correspondence using energy minimization and mutual information,” in Proceedings of IEEE International Conference on Computer Vision, (IEEE, 2003), pp. 1033–1040.

Knock, B. D.

F. Brackx, B. D. Knock, and H. D. Schepper, “Generalized multidimensional hilbert transforms in clifford analysis,” Int. J. Math. Math. Sci. 2006, 98145 (2006).
[CrossRef]

Kolmogorov, V.

J. Kim, V. Kolmogorov, and R. Zabih, “Visual correspondence using energy minimization and mutual information,” in Proceedings of IEEE International Conference on Computer Vision, (IEEE, 2003), pp. 1033–1040.

Lamanna, A.

C. Fookes, M. Bennamoun, and A. Lamanna, “Improved stereo image matching using mutual information and hierarchical prior probabilities,” in Proceedings of 16th International Conference on Pattern Recognition, (IEEE, 2002), pp. 937–940.

Li, J.

Mascarilla, L.

G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision 40, 269–284 (2011).
[CrossRef]

Moravec, H.

H. Moravec, “Toward automatic visual obstacle avoidance,” in Proceedings of 5th International Joint Conference on Artificial Intelligence, (Morgan Kaufmann, 1977), pp. 584–590.

Oppenheim, A. V.

A. V. Oppenheim, “The importance of phase in signals,” Proc. IEEE 69, 529–541 (1981).
[CrossRef]

Pal, C.

D. Scharstein and C. Pal, “Learning conditional random fields for stereo,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2007), pp. 1–8.

Roser, M.

A. Geiger, M. Roser, and R. Urtasun, “Efficient large-scale stereo matching,” in Proceedings of 10th Asian conference on Computer vision - Volume Part I, (Springer-Verlag, 2011), pp. 25–38.

Sangwine, S. J.

S. J. Sangwine, “Fourier transforms of color images using quaternion or hypercomplex numbers,” Electron. Lett. 32, 1979–1980 (1996).
[CrossRef]

N. L. Bihan and S. J. Sangwine, “Quaternion principal component analysis of color images,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2003), pp. 809–812.

Sarkar, I.

I. Sarkar and M. Bansal, “A wavelet-based multiresolution approach to solve the stereo correspondence problem using mutual information,” IEEE Trans. Syst. Man. Cybern., B: Cybern. 37, 1009–1014 (2007).
[CrossRef]

Scharstein, D.

D. Scharstein and C. Pal, “Learning conditional random fields for stereo,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2007), pp. 1–8.

Schepper, H. D.

F. Brackx, B. D. Knock, and H. D. Schepper, “Generalized multidimensional hilbert transforms in clifford analysis,” Int. J. Math. Math. Sci. 2006, 98145 (2006).
[CrossRef]

Sommer, G.

D. Zang and G. Sommer, “Signal modeling for two-dimensional image structures,” J. Visual Commun. Image 18, 81–99 (2007).
[CrossRef]

M. Felsberg and G. Sommer, “The monogenic scale-space: a unifying approach to phase-based image processing in scale-space,” J. Math. Imaging Vision 21, 5–26 (2004).
[CrossRef]

M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49, 3136–3144 (2001).
[CrossRef]

Tomasi, C.

S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 401–406 (1998).
[CrossRef]

Urtasun, R.

A. Geiger, M. Roser, and R. Urtasun, “Efficient large-scale stereo matching,” in Proceedings of 10th Asian conference on Computer vision - Volume Part I, (Springer-Verlag, 2011), pp. 25–38.

Veksler, O.

Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001).
[CrossRef]

Zabih, R.

Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001).
[CrossRef]

J. Kim, V. Kolmogorov, and R. Zabih, “Visual correspondence using energy minimization and mutual information,” in Proceedings of IEEE International Conference on Computer Vision, (IEEE, 2003), pp. 1033–1040.

Zang, D.

Zhang, D.

Electron. Lett. (1)

S. J. Sangwine, “Fourier transforms of color images using quaternion or hypercomplex numbers,” Electron. Lett. 32, 1979–1980 (1996).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (2)

Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001).
[CrossRef]

S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is insensitive to image sampling,” IEEE Trans. Pattern Anal. Mach. Intell. 20, 401–406 (1998).
[CrossRef]

IEEE Trans. Signal Process. (1)

M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49, 3136–3144 (2001).
[CrossRef]

IEEE Trans. Syst. Man. Cybern., B: Cybern. (1)

I. Sarkar and M. Bansal, “A wavelet-based multiresolution approach to solve the stereo correspondence problem using mutual information,” IEEE Trans. Syst. Man. Cybern., B: Cybern. 37, 1009–1014 (2007).
[CrossRef]

Int. J. Math. Math. Sci. (1)

F. Brackx, B. D. Knock, and H. D. Schepper, “Generalized multidimensional hilbert transforms in clifford analysis,” Int. J. Math. Math. Sci. 2006, 98145 (2006).
[CrossRef]

J. Math. Imaging Vision (2)

M. Felsberg and G. Sommer, “The monogenic scale-space: a unifying approach to phase-based image processing in scale-space,” J. Math. Imaging Vision 21, 5–26 (2004).
[CrossRef]

G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, “The color monogenic signal: application to color edge detection and color optical flow,” J. Math. Imaging Vision 40, 269–284 (2011).
[CrossRef]

J. Visual Commun. Image (1)

D. Zang and G. Sommer, “Signal modeling for two-dimensional image structures,” J. Visual Commun. Image 18, 81–99 (2007).
[CrossRef]

Opt. Lett. (1)

Proc. IEEE (1)

A. V. Oppenheim, “The importance of phase in signals,” Proc. IEEE 69, 529–541 (1981).
[CrossRef]

Other (7)

D. Scharstein and C. Pal, “Learning conditional random fields for stereo,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, (IEEE, 2007), pp. 1–8.

N. L. Bihan and S. J. Sangwine, “Quaternion principal component analysis of color images,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2003), pp. 809–812.

http://vision.middlebury.edu/stereo/ .

J. Kim, V. Kolmogorov, and R. Zabih, “Visual correspondence using energy minimization and mutual information,” in Proceedings of IEEE International Conference on Computer Vision, (IEEE, 2003), pp. 1033–1040.

A. Geiger, M. Roser, and R. Urtasun, “Efficient large-scale stereo matching,” in Proceedings of 10th Asian conference on Computer vision - Volume Part I, (Springer-Verlag, 2011), pp. 25–38.

H. Moravec, “Toward automatic visual obstacle avoidance,” in Proceedings of 5th International Joint Conference on Artificial Intelligence, (Morgan Kaufmann, 1977), pp. 584–590.

C. Fookes, M. Bennamoun, and A. Lamanna, “Improved stereo image matching using mutual information and hierarchical prior probabilities,” in Proceedings of 16th International Conference on Pattern Recognition, (IEEE, 2002), pp. 937–940.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1

Top row: Test images taken from [15]. Bottom row: Corresponding color monogenic curvature phase images.

Fig. 2
Fig. 2

Multiscale disparity estimation based on the color monogenic curvature phase.

Fig. 3
Fig. 3

Top row from left to right: Two views of “baby1” taken under different lighting and camera exposure conditions and the disparity ground truth. Bottom row from left to right: Estimated disparity maps based on [5], gray monogenic curvature phase [11] and the proposed method.

Fig. 4
Fig. 4

Top row from left to right: Two views of “lampshade1” taken under different lighting and camera exposure conditions and the disparity ground truth. Bottom row from left to right: Estimated disparity maps based on [5], gray monogenic curvature phase [11] and the proposed method.

Fig. 5
Fig. 5

Disparity errors in unoccluded areas with respect to different lighting combinations for “baby1” and “lampshade1”.

Fig. 6
Fig. 6

Disparity errors in unoccluded areas with respect to different signal to noise ratios for “baby1” and “lampshade1”.

Fig. 7
Fig. 7

Top row from left to right: Two views of outdoor images with large brightness change. Bottom row from left to right: Estimated disparity maps based on [5], gray monogenic curvature phase [11] and the proposed method.

Fig. 8
Fig. 8

Top row from left to right: Two views of outdoor images with large brightness change. Bottom row from left to right: Estimated disparity maps based on [5], gray monogenic curvature phase [11] and the proposed method.

Equations (20)

Equations on this page are rendered with MathJax. Learn more.

f m c = [ f 1 f 2 f 3 ] T ,
f 1 = [ ( f * h 1 ) * h 1 ] [ ( f * h 2 ) * h 2 ] [ ( f * h 1 ) * h 2 ] 2 ,
f 2 = 1 { F 1 cos 2 α } ,
f 3 = 1 { F 1 sin 2 α } ,
A ( x , y , s ) = f ( x , y , s ) 1 2 + f ( x , y , s ) 2 2 + f ( x , y , s ) 3 2 ,
θ ( x , y , s ) = 1 2 atan 2 ( f 3 ( x , y , s ) , f 2 ( x , y , s ) ) , θ ( π 2 , π 2 ] ,
Φ ( x , y , s ) = u ( x , y , s ) | u ( x , y , s ) | atan 2 ( | u ( x , y , s ) | , f 1 ( x , y , s ) ) , Φ ( π , π ] ,
f ( x , y ) = f r ( x , y ) i + f g ( x , y ) j + f b ( x , y ) k ,
f c m c = f r m c i + f g m c j + f b m c k with
f n m c = [ f n 1 ( x , y , s ) f n 2 ( x , y , s ) f n 3 ( x , y , s ) ] , n { r , g , b } ,
Φ c m c = Φ r i + Φ g j + Φ b k with
Φ n = u n ( x , y , s ) | u n ( x , y , s ) | atan 2 ( | u n ( x , y , s ) | , f n 1 ( x , y , s ) ) , Φ n ( π , π ] , n { r , g , b } .
CPMI ( Φ l , s , Φ r , s ) = H ( Φ l , s ) + H ( Φ r , s ) H ( Φ l , s , Φ r , s ) ,
H ( Φ ) = E Φ [ log ( P ( Φ ) ) ] = ϕ i Ω ϕ log ( P ( Φ = ϕ i ) ) P ( Φ = ϕ i ) ,
H ( Φ l , s , Φ r , s ) = E Φ l , s [ E Φ r , s [ log ( P ( Φ l , s , Φ r , s ) ) ] ] ,
CPMI ( Φ l , s , Φ r , s ) p cpmi ( Φ l , s ( p ) , Φ r , s ( p + d p ) ) ,
E = E data + E smooth ,
E data = p cpmi ( Φ l , s ( p ) , Φ r , s ( p + d p ) ) .
E smooth = p q 𝒩 ( p ) V p q ( d p , d q ) ,
V p q ( d p , d q ) = λ min ( | d p d q | 2 , V max ) ,

Metrics