Abstract

This study analyzed the implementation and performance of a framework that can be efficiently applied to three-dimensional (3D) video sequence visualization. The proposed algorithm is based on wavelets and wavelet atomic functions used in the computation of disparity maps. The proposed algorithm employs wavelet multilevel decomposition and 3D visualization via color anaglyphs synthesis. Simulations were run on synthetic images, synthetic video sequences, and real-life video sequences. Results shows that this novel approach performs better in depth and spatial perception tasks compared to existing methods, both in terms of objective criteria such as quantity of bad disparities and similarity structural index measure and the more subjective measure of human vision.

© 2011 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. J. L. Barron, D. J. Fleet, S. Beauchemin, and T. Burkitt, “Performance of optical flow techniques,” in Conference on Computer Vision and Pattern Recognition (IEEE, 1992), pp. 236–242.
  2. D. Fleet and A. Jepson, Measurement of Image Velocity (Kluwer, 1992).
    [CrossRef]
  3. B. B. Alagoz, “Obtaining depth maps from color images by region based stereo matching algorithms,” OncuBilim Algorithm and Systems Labs 8, 1–12 (2008).
  4. A. Bhatti and S. Nahavandi, 2008 Stereo Vision (I-Tech, 2008).
  5. E. Dubois, X. Huang, “3D reconstruction based on a hybrid disparity estimation algorithm,” in International Conference on Image Processing (IEEE, 2006), pp. 1025–1028.
  6. Y. Meyer, Ondelettes (Hermann, 1991).
  7. C. Juarez, V. Ponomaryov, J. Sanchez, and V. Kravchenko, “Wavelets based on atomic function used in detection and classification of masses in mammography,” in Lecture Notes in Artificial Intelligence (Springer, 2008), Vol.  LNAI 5317, pp. 295–304.
  8. V. Ponomaryov and E. Ramos, “3D video sequence reconstruction algorithms implemented on a DSP,” Proc. SPIE 7871, 78711D (2011).
    [CrossRef]
  9. V. Kravchenko, H. Meana, V. Ponomaryov, 2009 Adaptive Digital Processing of Multidimensional Signals with Applications (FizMatLit, 2009). Available at http://www.posgrados.esimecu.ipn.mx/.
  10. Y. Gulayev, V. Kravchenko, and V. Pustoviot, “A new class of WA-systems of Kravchenko-Rvachev functions,” Dokl., Math. 75, 325–332 (2007).
    [CrossRef]
  11. M. Unser and A. Aldroubi, “A general sampling theorem for nonideal acquisition devices,” IEEE Trans. Signal Process. 42, 2915–2925 (1994).
    [CrossRef]
  12. W. Sanders and D. McAllister, “Producing anaglyphs from synthetic images,” Proc. SPIE 5006, 348–358 (2003).
    [CrossRef]
  13. I. Ideses and L. Yaroslavsky, “Three methods that improve the visual quality of color anaglyphs,” J. Opt. A 7, 755–762(2005).
    [CrossRef]
  14. P. Yaroslavsky, J. Campos, M. Espínola, and I. Ideses, “Redundancy of stereoscopic images: experimental evaluation,” Opt. Express 13, 10895–10907 (2005).
    [CrossRef] [PubMed]
  15. I. Idesses, L. Yaroslavsky, and B. Fishbain, “3D from compressed 2D video,” Proc. SPIE 6490, 64901C (2007).
    [CrossRef]
  16. I. Idesses and L. Yaroslavsky, “A method for generating 3D video from a single video stream,” in VMV (Aka GmbH, 2002), pp. 435–438.
  17. Middlebury College, “Middlebury stereo datasets,” http://vision.middlebury.edu/stereo/data.
  18. Arizona State University, “YUV video sequences,” http://trace.eas.asu.edu/yuv/index.html.
  19. W. S. Malpica and A. C. Bovik, “Range image quality assessment by structural similarity,” in IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2009), pp. 1149–1152.
    [CrossRef]
  20. Z. Wang and A. C. Bovik, “Mean squared error: love it or leave it? A new look at signal fidelity measures,” IEEE Signal Process. Mag. 26, 98–117 (2009).
    [CrossRef]

2011 (1)

V. Ponomaryov and E. Ramos, “3D video sequence reconstruction algorithms implemented on a DSP,” Proc. SPIE 7871, 78711D (2011).
[CrossRef]

2009 (1)

Z. Wang and A. C. Bovik, “Mean squared error: love it or leave it? A new look at signal fidelity measures,” IEEE Signal Process. Mag. 26, 98–117 (2009).
[CrossRef]

2008 (1)

B. B. Alagoz, “Obtaining depth maps from color images by region based stereo matching algorithms,” OncuBilim Algorithm and Systems Labs 8, 1–12 (2008).

2007 (2)

Y. Gulayev, V. Kravchenko, and V. Pustoviot, “A new class of WA-systems of Kravchenko-Rvachev functions,” Dokl., Math. 75, 325–332 (2007).
[CrossRef]

I. Idesses, L. Yaroslavsky, and B. Fishbain, “3D from compressed 2D video,” Proc. SPIE 6490, 64901C (2007).
[CrossRef]

2005 (2)

I. Ideses and L. Yaroslavsky, “Three methods that improve the visual quality of color anaglyphs,” J. Opt. A 7, 755–762(2005).
[CrossRef]

P. Yaroslavsky, J. Campos, M. Espínola, and I. Ideses, “Redundancy of stereoscopic images: experimental evaluation,” Opt. Express 13, 10895–10907 (2005).
[CrossRef] [PubMed]

2003 (1)

W. Sanders and D. McAllister, “Producing anaglyphs from synthetic images,” Proc. SPIE 5006, 348–358 (2003).
[CrossRef]

1994 (1)

M. Unser and A. Aldroubi, “A general sampling theorem for nonideal acquisition devices,” IEEE Trans. Signal Process. 42, 2915–2925 (1994).
[CrossRef]

Alagoz, B. B.

B. B. Alagoz, “Obtaining depth maps from color images by region based stereo matching algorithms,” OncuBilim Algorithm and Systems Labs 8, 1–12 (2008).

Aldroubi, A.

M. Unser and A. Aldroubi, “A general sampling theorem for nonideal acquisition devices,” IEEE Trans. Signal Process. 42, 2915–2925 (1994).
[CrossRef]

Barron, J. L.

J. L. Barron, D. J. Fleet, S. Beauchemin, and T. Burkitt, “Performance of optical flow techniques,” in Conference on Computer Vision and Pattern Recognition (IEEE, 1992), pp. 236–242.

Beauchemin, S.

J. L. Barron, D. J. Fleet, S. Beauchemin, and T. Burkitt, “Performance of optical flow techniques,” in Conference on Computer Vision and Pattern Recognition (IEEE, 1992), pp. 236–242.

Bhatti, A.

A. Bhatti and S. Nahavandi, 2008 Stereo Vision (I-Tech, 2008).

Bovik, A. C.

Z. Wang and A. C. Bovik, “Mean squared error: love it or leave it? A new look at signal fidelity measures,” IEEE Signal Process. Mag. 26, 98–117 (2009).
[CrossRef]

W. S. Malpica and A. C. Bovik, “Range image quality assessment by structural similarity,” in IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2009), pp. 1149–1152.
[CrossRef]

Burkitt, T.

J. L. Barron, D. J. Fleet, S. Beauchemin, and T. Burkitt, “Performance of optical flow techniques,” in Conference on Computer Vision and Pattern Recognition (IEEE, 1992), pp. 236–242.

Campos, J.

Dubois, E.

E. Dubois, X. Huang, “3D reconstruction based on a hybrid disparity estimation algorithm,” in International Conference on Image Processing (IEEE, 2006), pp. 1025–1028.

Espínola, M.

Fishbain, B.

I. Idesses, L. Yaroslavsky, and B. Fishbain, “3D from compressed 2D video,” Proc. SPIE 6490, 64901C (2007).
[CrossRef]

Fleet, D.

D. Fleet and A. Jepson, Measurement of Image Velocity (Kluwer, 1992).
[CrossRef]

Fleet, D. J.

J. L. Barron, D. J. Fleet, S. Beauchemin, and T. Burkitt, “Performance of optical flow techniques,” in Conference on Computer Vision and Pattern Recognition (IEEE, 1992), pp. 236–242.

Gulayev, Y.

Y. Gulayev, V. Kravchenko, and V. Pustoviot, “A new class of WA-systems of Kravchenko-Rvachev functions,” Dokl., Math. 75, 325–332 (2007).
[CrossRef]

Huang, X.

E. Dubois, X. Huang, “3D reconstruction based on a hybrid disparity estimation algorithm,” in International Conference on Image Processing (IEEE, 2006), pp. 1025–1028.

Ideses, I.

P. Yaroslavsky, J. Campos, M. Espínola, and I. Ideses, “Redundancy of stereoscopic images: experimental evaluation,” Opt. Express 13, 10895–10907 (2005).
[CrossRef] [PubMed]

I. Ideses and L. Yaroslavsky, “Three methods that improve the visual quality of color anaglyphs,” J. Opt. A 7, 755–762(2005).
[CrossRef]

Idesses, I.

I. Idesses, L. Yaroslavsky, and B. Fishbain, “3D from compressed 2D video,” Proc. SPIE 6490, 64901C (2007).
[CrossRef]

I. Idesses and L. Yaroslavsky, “A method for generating 3D video from a single video stream,” in VMV (Aka GmbH, 2002), pp. 435–438.

Jepson, A.

D. Fleet and A. Jepson, Measurement of Image Velocity (Kluwer, 1992).
[CrossRef]

Juarez, C.

C. Juarez, V. Ponomaryov, J. Sanchez, and V. Kravchenko, “Wavelets based on atomic function used in detection and classification of masses in mammography,” in Lecture Notes in Artificial Intelligence (Springer, 2008), Vol.  LNAI 5317, pp. 295–304.

Kravchenko, V.

Y. Gulayev, V. Kravchenko, and V. Pustoviot, “A new class of WA-systems of Kravchenko-Rvachev functions,” Dokl., Math. 75, 325–332 (2007).
[CrossRef]

C. Juarez, V. Ponomaryov, J. Sanchez, and V. Kravchenko, “Wavelets based on atomic function used in detection and classification of masses in mammography,” in Lecture Notes in Artificial Intelligence (Springer, 2008), Vol.  LNAI 5317, pp. 295–304.

V. Kravchenko, H. Meana, V. Ponomaryov, 2009 Adaptive Digital Processing of Multidimensional Signals with Applications (FizMatLit, 2009). Available at http://www.posgrados.esimecu.ipn.mx/.

Malpica, W. S.

W. S. Malpica and A. C. Bovik, “Range image quality assessment by structural similarity,” in IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2009), pp. 1149–1152.
[CrossRef]

McAllister, D.

W. Sanders and D. McAllister, “Producing anaglyphs from synthetic images,” Proc. SPIE 5006, 348–358 (2003).
[CrossRef]

Meana, H.

V. Kravchenko, H. Meana, V. Ponomaryov, 2009 Adaptive Digital Processing of Multidimensional Signals with Applications (FizMatLit, 2009). Available at http://www.posgrados.esimecu.ipn.mx/.

Meyer, Y.

Y. Meyer, Ondelettes (Hermann, 1991).

Nahavandi, S.

A. Bhatti and S. Nahavandi, 2008 Stereo Vision (I-Tech, 2008).

Ponomaryov, V.

V. Ponomaryov and E. Ramos, “3D video sequence reconstruction algorithms implemented on a DSP,” Proc. SPIE 7871, 78711D (2011).
[CrossRef]

V. Kravchenko, H. Meana, V. Ponomaryov, 2009 Adaptive Digital Processing of Multidimensional Signals with Applications (FizMatLit, 2009). Available at http://www.posgrados.esimecu.ipn.mx/.

C. Juarez, V. Ponomaryov, J. Sanchez, and V. Kravchenko, “Wavelets based on atomic function used in detection and classification of masses in mammography,” in Lecture Notes in Artificial Intelligence (Springer, 2008), Vol.  LNAI 5317, pp. 295–304.

Pustoviot, V.

Y. Gulayev, V. Kravchenko, and V. Pustoviot, “A new class of WA-systems of Kravchenko-Rvachev functions,” Dokl., Math. 75, 325–332 (2007).
[CrossRef]

Ramos, E.

V. Ponomaryov and E. Ramos, “3D video sequence reconstruction algorithms implemented on a DSP,” Proc. SPIE 7871, 78711D (2011).
[CrossRef]

Sanchez, J.

C. Juarez, V. Ponomaryov, J. Sanchez, and V. Kravchenko, “Wavelets based on atomic function used in detection and classification of masses in mammography,” in Lecture Notes in Artificial Intelligence (Springer, 2008), Vol.  LNAI 5317, pp. 295–304.

Sanders, W.

W. Sanders and D. McAllister, “Producing anaglyphs from synthetic images,” Proc. SPIE 5006, 348–358 (2003).
[CrossRef]

Unser, M.

M. Unser and A. Aldroubi, “A general sampling theorem for nonideal acquisition devices,” IEEE Trans. Signal Process. 42, 2915–2925 (1994).
[CrossRef]

Wang, Z.

Z. Wang and A. C. Bovik, “Mean squared error: love it or leave it? A new look at signal fidelity measures,” IEEE Signal Process. Mag. 26, 98–117 (2009).
[CrossRef]

Yaroslavsky, L.

I. Idesses, L. Yaroslavsky, and B. Fishbain, “3D from compressed 2D video,” Proc. SPIE 6490, 64901C (2007).
[CrossRef]

I. Ideses and L. Yaroslavsky, “Three methods that improve the visual quality of color anaglyphs,” J. Opt. A 7, 755–762(2005).
[CrossRef]

I. Idesses and L. Yaroslavsky, “A method for generating 3D video from a single video stream,” in VMV (Aka GmbH, 2002), pp. 435–438.

Yaroslavsky, P.

Dokl., Math. (1)

Y. Gulayev, V. Kravchenko, and V. Pustoviot, “A new class of WA-systems of Kravchenko-Rvachev functions,” Dokl., Math. 75, 325–332 (2007).
[CrossRef]

IEEE Signal Process. Mag. (1)

Z. Wang and A. C. Bovik, “Mean squared error: love it or leave it? A new look at signal fidelity measures,” IEEE Signal Process. Mag. 26, 98–117 (2009).
[CrossRef]

IEEE Trans. Signal Process. (1)

M. Unser and A. Aldroubi, “A general sampling theorem for nonideal acquisition devices,” IEEE Trans. Signal Process. 42, 2915–2925 (1994).
[CrossRef]

J. Opt. A (1)

I. Ideses and L. Yaroslavsky, “Three methods that improve the visual quality of color anaglyphs,” J. Opt. A 7, 755–762(2005).
[CrossRef]

OncuBilim Algorithm and Systems Labs (1)

B. B. Alagoz, “Obtaining depth maps from color images by region based stereo matching algorithms,” OncuBilim Algorithm and Systems Labs 8, 1–12 (2008).

Opt. Express (1)

Proc. SPIE (3)

W. Sanders and D. McAllister, “Producing anaglyphs from synthetic images,” Proc. SPIE 5006, 348–358 (2003).
[CrossRef]

I. Idesses, L. Yaroslavsky, and B. Fishbain, “3D from compressed 2D video,” Proc. SPIE 6490, 64901C (2007).
[CrossRef]

V. Ponomaryov and E. Ramos, “3D video sequence reconstruction algorithms implemented on a DSP,” Proc. SPIE 7871, 78711D (2011).
[CrossRef]

Other (11)

V. Kravchenko, H. Meana, V. Ponomaryov, 2009 Adaptive Digital Processing of Multidimensional Signals with Applications (FizMatLit, 2009). Available at http://www.posgrados.esimecu.ipn.mx/.

J. L. Barron, D. J. Fleet, S. Beauchemin, and T. Burkitt, “Performance of optical flow techniques,” in Conference on Computer Vision and Pattern Recognition (IEEE, 1992), pp. 236–242.

D. Fleet and A. Jepson, Measurement of Image Velocity (Kluwer, 1992).
[CrossRef]

A. Bhatti and S. Nahavandi, 2008 Stereo Vision (I-Tech, 2008).

E. Dubois, X. Huang, “3D reconstruction based on a hybrid disparity estimation algorithm,” in International Conference on Image Processing (IEEE, 2006), pp. 1025–1028.

Y. Meyer, Ondelettes (Hermann, 1991).

C. Juarez, V. Ponomaryov, J. Sanchez, and V. Kravchenko, “Wavelets based on atomic function used in detection and classification of masses in mammography,” in Lecture Notes in Artificial Intelligence (Springer, 2008), Vol.  LNAI 5317, pp. 295–304.

I. Idesses and L. Yaroslavsky, “A method for generating 3D video from a single video stream,” in VMV (Aka GmbH, 2002), pp. 435–438.

Middlebury College, “Middlebury stereo datasets,” http://vision.middlebury.edu/stereo/data.

Arizona State University, “YUV video sequences,” http://trace.eas.asu.edu/yuv/index.html.

W. S. Malpica and A. C. Bovik, “Range image quality assessment by structural similarity,” in IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2009), pp. 1149–1152.
[CrossRef]

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1

Block diagram of the proposed algorithm.

Fig. 2
Fig. 2

Frequency filter responses, (a) LP, (b) HP.

Fig. 3
Fig. 3

Block diagram of the algorithm for the disparity map computation.

Fig. 4
Fig. 4

Disparity maps obtained using M-WAF π 6 for synthetic images (a) Aloe, (b) Venus, (c) Lampshade1, and (d) Wood1, respectively.

Fig. 5
Fig. 5

Synthesized anaglyphs using M-WAF π 6 in (a) Aloe, (b) Venus, (c) Lampshade1, and (d) Wood1 images.

Fig. 6
Fig. 6

Synthesized anaglyphs using: M-WAF π 6 in (a) Coastguard, (b) Flowers, and (c) Video Test, respectively.

Tables (1)

Tables Icon

Table 1 QBD and SSID Metrics Values for Different Synthetic Stereo Pairs

Equations (17)

Equations on this page are rendered with MathJax. Learn more.

W i = | W i | exp ( j Θ i ) ,
| W i | = ( | D h , i | 2 + | D v , i | 2 + | D d , i | 2 ) 1 2 ,
Θ i = { α i if     D h , i > 0 π α i if     D h , i < 0 , α i = tan 1 ( D v , i / D h , i ) .
COR i ( x , y ) = ( a , b ) Q W i , left ( a , b ) · W i , right * ( x + a , y + b ) ( a , b Q | W | i , left 2 ( a , b ) · a , b Q | W | i , right 2 ( x + a , y + b ) ) 1 2 ,
ψ θ p ( x ) = exp j π x h θ p ( x ) h θ p ( x ) ,
h θ p ( x ) = 1 2 p k = 0 ( p 1 ) / 2 C p k ( θ ( x + p 2 k 2 ) + θ ( x p 2 k 2 ) ) odd     p ,
h θ p ( x ) = 1 2 p [ k = 0 ( p 2 ) / 2 C p k ( θ ( x + p 2 k 2 ) + θ ( x p 2 k 2 ) ) + C p p / 2 θ ( x ) ] even     p ,
u p ( x ) = 1 2 π e j u x k = 1 sin ( u · 2 k ) u · 2 k d u .
u p m ( x ) = 1 2 π e j u x k = 1 sin 2 ( m u ( 2 m ) k ) m u ( 2 m ) k sin ( u ( 2 m ) k ) d u , m = 1 , 2 , 3
f u p N ( x ) = 1 2 π e j u x ( sin ( u 2 ) u 2 ) N k = 1 sin ( u · 2 k ) u · 2 k d u .
π m ( t ) = 1 2 π e i x t Π m ( t ) d t , Π m ( x ) = k = 1 m [ sin ( 2 m 1 ) ω / ( 2 m ) k + ν = 2 m ( 1 ) ν sin ( 2 m 2 ν + 1 ) ω / ( 2 m ) ν ] ( 3 m 2 ) ω / ( 2 m ) k .
D new = a · D P ,
QBD = 1 N x , y | d E ( x , y ) d G ( x , y ) | 2 ,
SSIM ( x , y ) = [ l ( x , y ) ] · [ c ( x , y ) ] · [ s ( x , y ) ] ,
l ( x , y ) = 2 μ X ( x , y ) μ Y ( x , y ) + C 1 μ X 2 ( x , y ) + μ Y 2 ( x , y ) + C 1 ,
c ( x , y ) = 2 σ X ( x , y ) σ Y ( x , y ) + C 2 σ X 2 ( x , y ) + σ Y 2 ( x , y ) + C 2 ,
s ( x , y ) = σ X Y ( x , y ) + C 3 σ X ( x , y ) + σ Y ( x , y ) + C 3 ,

Metrics