Video stabilization in atmosphere turbulent conditions based on the Laplacian-Riesz pyramid

Bindang Xue; Yi Liu; Linyan Cui; Xiangzhi Bai; Xiaoguang Cao; Fugen Zhou

doi:10.1364/OE.24.028092

Abstract

Video stabilization in atmosphere turbulent conditions is aimed at removing spatiotemporally varying distortions from video recordings. Conventional shaky video stabilization approaches do not perform effectively under turbulent circumstances due to the erratic motion common to those conditions. Using complex-valued image pyramids, we propose a method to mitigate this erratic motion in videos. First, each frame of a video is decomposed into different spatial frequencies using the Laplacian pyramid. Second, a Riesz transform is adopted to extract the local amplitude and the local phase of each sub-band. Next, low-pass filters are designed to attenuate the local amplitude and phase variations to remove turbulence-induced distortions. Experimental results show that the proposed approach is efficient and provides stabilizing video in atmosphere turbulent conditions.

Full Article | PDF Article

OSA Recommended Articles

Geometric correction of atmospheric turbulence-degraded video containing moving objects

Kalyan Kumar Halder, Murat Tahtali, and Sreenatha G. Anavatti
Opt. Express 23(4) 5091-5101 (2015)

Restoring atmospheric-turbulence-degraded images

Md. Hasan Furhad, Murat Tahtali, and Andrew Lambert
Appl. Opt. 55(19) 5082-5090 (2016)

Multi-focus image fusion algorithm based on Laplacian pyramids

Jianguo Sun, Qilong Han, Liang Kou, Liguo Zhang, Kejia Zhang, and Zilong Jin
J. Opt. Soc. Am. A 35(3) 480-490 (2018)

References

View by:
Article Order
|
Year
|
Author
|
Publication

R. K. Tyson, Principles of Adaptive Optics (Academic, 1991).
Institute of Astronomy, University of Cambridge, “Lucky Imaging”, http://www.ast.cam.ac.uk/~optics/Lucky_Web_Site/index.htm
M. Loktev, O. Soloviev, S. Savenko, and G. Vdovin, “Speckle imaging through turbulent atmosphere based on adaptable pupil segmentation,” Opt. Lett. 36(14), 2656–2658 (2011).
[Crossref] [PubMed]
D. A. Hope, S. M. Jefferies, M. Hart, and J. G. Nagy, “High-resolution speckle imaging through strong atmospheric turbulence,” Opt. Express 24(11), 12116–12129 (2016).
[Crossref] [PubMed]
D. Li, R. M. Mersereau, and S. Simske, “Atmospheric turbulence-degraded image restoration using principal components analysis,” IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007).
[Crossref]
X. Zhu and P. Milanfar, “Image reconstruction from videos distorted by atmospheric turbulence,” Proc. SPIE 7543(1), 423–426 (2010).
R. Gal, N. Kiryati, and N. Sochen, “Progress in the restoration of image sequences degraded by atmospheric turbulence,” Pattern Recognit. Lett. 48, 8–14 (2014).
[Crossref]
N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. R. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22(6), 2398–2408 (2013).
[Crossref] [PubMed]
L. Yaroslavsky, B. Fishbain, G. Shabat, and I. Ideses, “Superresolution in turbulent videos: making profit from damage,” Opt. Lett. 32(20), 3038–3040 (2007).
[Crossref] [PubMed]
O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 450–462 (2013).
[Crossref] [PubMed]
Y. Lou, S. H. Kang, S. Soatto, and A. L. Bertozzi, “Video stabilization of atmospheric turbulence distortion,” Inverse Probl. Imaging (Springfield) 7(3), 839–861 (2013).
[Crossref]
N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Phase-based video motion processing,” ACM TOG 32(4), 80 (2013).
[Crossref]
A. V. Oppenheim and J. S. Lim, “The importance of phase in signals,” InProceedings of the IEEE (IEEE, 1981), 69(5), pp, 529–541.
M. C. Morrone and R. A. Owens, “Feature detection from local energy,” Pattern Recognit. Lett. 6(5), 303–313 (1987).
[Crossref]
P. Kovesi, “Image features from phase congruency,” J. Computer Vis. Res. 1(3), 1–26 (1999).
F. J. Madrid-Cuevas, R. Medina-Carnicer, Á. Carmona-Poyato, and N. L. Fernández-García, “Dominant Points Detection Using Phase Congruence,” Iberian Conference on Pattern Recognition and Image Analysis (Springer Berlin Heidelberg, 2007), pp.138–145.
[Crossref]
N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Riesz pyramids for fast phase-based video magnification,” In proceedings of ICCP, (IEEE, 2014), pp.1–10.
M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49(12), 3136–3144 (2001).
[Crossref]
P. J. Burt and E. H. Shashua, “The laplacian pyramid as a compact image code,” IEEE Trans. Commun. 31(4), 532–540 (1983).
[Crossref]
N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “SIGGRAPH 2013 Phase-Based Video Motion Processing”, http://people.csail.mit.edu/nwadhwa/phase-video/
E. Repasi and R. Weiss, “Analysis of Image Distortions by Atmospheric Turbulence and Computer Simulation of Turbulence Effects,” Proc. SPIE 6941, 69410S (2008).
[Crossref]
E. Repasi and R. Weiss, “Computer Simulation of Image Degradations by Atmospheric Turbulence for Horizontal Views,” Proc. SPIE 8014, 80140U (2011).
[Crossref]
O. Haik and Y. Yitzhaky, http://www.ee.bgu.ac.il/~itzik/VideosOE06/
I. Dror and N. S. Kopeika, “Experimental comparison of turbulence MTF and aerosol MTF through open atmosphere,” J. Opt. Soc. Am. A 12(5), 970–980 (1995).
[Crossref]
X. Zhu and P. Milanfar, “Removing atmospheric Turbulence via Space-Invariant Deconvolution,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 157–170 (2013).
[Crossref] [PubMed]
P. Ruiz, X. Zhou, J. Mateos, R. Molina, and A. K. Katsaggelos, “Variational Bayesian Blind Image Deconvolution: A review,” Digit. Signal Process. 47, 116–127 (2015).
[Crossref]

2016 (1)

D. A. Hope, S. M. Jefferies, M. Hart, and J. G. Nagy, “High-resolution speckle imaging through strong atmospheric turbulence,” Opt. Express 24(11), 12116–12129 (2016).
[Crossref] [PubMed]

2015 (1)

P. Ruiz, X. Zhou, J. Mateos, R. Molina, and A. K. Katsaggelos, “Variational Bayesian Blind Image Deconvolution: A review,” Digit. Signal Process. 47, 116–127 (2015).
[Crossref]

2014 (1)

R. Gal, N. Kiryati, and N. Sochen, “Progress in the restoration of image sequences degraded by atmospheric turbulence,” Pattern Recognit. Lett. 48, 8–14 (2014).
[Crossref]

2013 (5)

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. R. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22(6), 2398–2408 (2013).
[Crossref] [PubMed]

X. Zhu and P. Milanfar, “Removing atmospheric Turbulence via Space-Invariant Deconvolution,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 157–170 (2013).
[Crossref] [PubMed]

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 450–462 (2013).
[Crossref] [PubMed]

Y. Lou, S. H. Kang, S. Soatto, and A. L. Bertozzi, “Video stabilization of atmospheric turbulence distortion,” Inverse Probl. Imaging (Springfield) 7(3), 839–861 (2013).
[Crossref]

N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Phase-based video motion processing,” ACM TOG 32(4), 80 (2013).
[Crossref]

2011 (2)

E. Repasi and R. Weiss, “Computer Simulation of Image Degradations by Atmospheric Turbulence for Horizontal Views,” Proc. SPIE 8014, 80140U (2011).
[Crossref]

M. Loktev, O. Soloviev, S. Savenko, and G. Vdovin, “Speckle imaging through turbulent atmosphere based on adaptable pupil segmentation,” Opt. Lett. 36(14), 2656–2658 (2011).
[Crossref] [PubMed]

2010 (1)

X. Zhu and P. Milanfar, “Image reconstruction from videos distorted by atmospheric turbulence,” Proc. SPIE 7543(1), 423–426 (2010).

2008 (1)

E. Repasi and R. Weiss, “Analysis of Image Distortions by Atmospheric Turbulence and Computer Simulation of Turbulence Effects,” Proc. SPIE 6941, 69410S (2008).
[Crossref]

2007 (2)

L. Yaroslavsky, B. Fishbain, G. Shabat, and I. Ideses, “Superresolution in turbulent videos: making profit from damage,” Opt. Lett. 32(20), 3038–3040 (2007).
[Crossref] [PubMed]

D. Li, R. M. Mersereau, and S. Simske, “Atmospheric turbulence-degraded image restoration using principal components analysis,” IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007).
[Crossref]

2001 (1)

M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49(12), 3136–3144 (2001).
[Crossref]

1999 (1)

P. Kovesi, “Image features from phase congruency,” J. Computer Vis. Res. 1(3), 1–26 (1999).

1995 (1)

I. Dror and N. S. Kopeika, “Experimental comparison of turbulence MTF and aerosol MTF through open atmosphere,” J. Opt. Soc. Am. A 12(5), 970–980 (1995).
[Crossref]

1987 (1)

M. C. Morrone and R. A. Owens, “Feature detection from local energy,” Pattern Recognit. Lett. 6(5), 303–313 (1987).
[Crossref]

1983 (1)

P. J. Burt and E. H. Shashua, “The laplacian pyramid as a compact image code,” IEEE Trans. Commun. 31(4), 532–540 (1983).
[Crossref]

Achim, A.

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. R. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22(6), 2398–2408 (2013).
[Crossref] [PubMed]

Anantrasirichai, N.

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. R. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22(6), 2398–2408 (2013).
[Crossref] [PubMed]

Bertozzi, A. L.

Y. Lou, S. H. Kang, S. Soatto, and A. L. Bertozzi, “Video stabilization of atmospheric turbulence distortion,” Inverse Probl. Imaging (Springfield) 7(3), 839–861 (2013).
[Crossref]

Bull, D. R.

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. R. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22(6), 2398–2408 (2013).
[Crossref] [PubMed]

Burt, P. J.

P. J. Burt and E. H. Shashua, “The laplacian pyramid as a compact image code,” IEEE Trans. Commun. 31(4), 532–540 (1983).
[Crossref]

Dror, I.

I. Dror and N. S. Kopeika, “Experimental comparison of turbulence MTF and aerosol MTF through open atmosphere,” J. Opt. Soc. Am. A 12(5), 970–980 (1995).
[Crossref]

Durand, F.

N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Phase-based video motion processing,” ACM TOG 32(4), 80 (2013).
[Crossref]

Felsberg, M.

M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49(12), 3136–3144 (2001).
[Crossref]

Fishbain, B.

L. Yaroslavsky, B. Fishbain, G. Shabat, and I. Ideses, “Superresolution in turbulent videos: making profit from damage,” Opt. Lett. 32(20), 3038–3040 (2007).
[Crossref] [PubMed]

Freeman, W. T.

N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Phase-based video motion processing,” ACM TOG 32(4), 80 (2013).
[Crossref]

Gal, R.

R. Gal, N. Kiryati, and N. Sochen, “Progress in the restoration of image sequences degraded by atmospheric turbulence,” Pattern Recognit. Lett. 48, 8–14 (2014).
[Crossref]

Kang, S. H.

Y. Lou, S. H. Kang, S. Soatto, and A. L. Bertozzi, “Video stabilization of atmospheric turbulence distortion,” Inverse Probl. Imaging (Springfield) 7(3), 839–861 (2013).
[Crossref]

Katsaggelos, A. K.

P. Ruiz, X. Zhou, J. Mateos, R. Molina, and A. K. Katsaggelos, “Variational Bayesian Blind Image Deconvolution: A review,” Digit. Signal Process. 47, 116–127 (2015).
[Crossref]

Kingsbury, N. G.

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. R. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22(6), 2398–2408 (2013).
[Crossref] [PubMed]

Kiryati, N.

R. Gal, N. Kiryati, and N. Sochen, “Progress in the restoration of image sequences degraded by atmospheric turbulence,” Pattern Recognit. Lett. 48, 8–14 (2014).
[Crossref]

Kopeika, N. S.

I. Dror and N. S. Kopeika, “Experimental comparison of turbulence MTF and aerosol MTF through open atmosphere,” J. Opt. Soc. Am. A 12(5), 970–980 (1995).
[Crossref]

Kovesi, P.

P. Kovesi, “Image features from phase congruency,” J. Computer Vis. Res. 1(3), 1–26 (1999).

Li, D.

D. Li, R. M. Mersereau, and S. Simske, “Atmospheric turbulence-degraded image restoration using principal components analysis,” IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007).
[Crossref]

Li, X.

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 450–462 (2013).
[Crossref] [PubMed]

Lim, J. S.

A. V. Oppenheim and J. S. Lim, “The importance of phase in signals,” InProceedings of the IEEE (IEEE, 1981), 69(5), pp, 529–541.

Loktev, M.

M. Loktev, O. Soloviev, S. Savenko, and G. Vdovin, “Speckle imaging through turbulent atmosphere based on adaptable pupil segmentation,” Opt. Lett. 36(14), 2656–2658 (2011).
[Crossref] [PubMed]

Lou, Y.

Y. Lou, S. H. Kang, S. Soatto, and A. L. Bertozzi, “Video stabilization of atmospheric turbulence distortion,” Inverse Probl. Imaging (Springfield) 7(3), 839–861 (2013).
[Crossref]

Mateos, J.

P. Ruiz, X. Zhou, J. Mateos, R. Molina, and A. K. Katsaggelos, “Variational Bayesian Blind Image Deconvolution: A review,” Digit. Signal Process. 47, 116–127 (2015).
[Crossref]

Mersereau, R. M.

D. Li, R. M. Mersereau, and S. Simske, “Atmospheric turbulence-degraded image restoration using principal components analysis,” IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007).
[Crossref]

Milanfar, P.

X. Zhu and P. Milanfar, “Removing atmospheric Turbulence via Space-Invariant Deconvolution,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 157–170 (2013).
[Crossref] [PubMed]

X. Zhu and P. Milanfar, “Image reconstruction from videos distorted by atmospheric turbulence,” Proc. SPIE 7543(1), 423–426 (2010).

Molina, R.

P. Ruiz, X. Zhou, J. Mateos, R. Molina, and A. K. Katsaggelos, “Variational Bayesian Blind Image Deconvolution: A review,” Digit. Signal Process. 47, 116–127 (2015).
[Crossref]

Morrone, M. C.

M. C. Morrone and R. A. Owens, “Feature detection from local energy,” Pattern Recognit. Lett. 6(5), 303–313 (1987).
[Crossref]

Nagy, J. G.

D. A. Hope, S. M. Jefferies, M. Hart, and J. G. Nagy, “High-resolution speckle imaging through strong atmospheric turbulence,” Opt. Express 24(11), 12116–12129 (2016).
[Crossref] [PubMed]

Oppenheim, A. V.

A. V. Oppenheim and J. S. Lim, “The importance of phase in signals,” InProceedings of the IEEE (IEEE, 1981), 69(5), pp, 529–541.

Oreifej, O.

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 450–462 (2013).
[Crossref] [PubMed]

Owens, R. A.

M. C. Morrone and R. A. Owens, “Feature detection from local energy,” Pattern Recognit. Lett. 6(5), 303–313 (1987).
[Crossref]

Repasi, E.

E. Repasi and R. Weiss, “Computer Simulation of Image Degradations by Atmospheric Turbulence for Horizontal Views,” Proc. SPIE 8014, 80140U (2011).
[Crossref]

E. Repasi and R. Weiss, “Analysis of Image Distortions by Atmospheric Turbulence and Computer Simulation of Turbulence Effects,” Proc. SPIE 6941, 69410S (2008).
[Crossref]

Rubinstein, M.

N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Phase-based video motion processing,” ACM TOG 32(4), 80 (2013).
[Crossref]

Ruiz, P.

P. Ruiz, X. Zhou, J. Mateos, R. Molina, and A. K. Katsaggelos, “Variational Bayesian Blind Image Deconvolution: A review,” Digit. Signal Process. 47, 116–127 (2015).
[Crossref]

Savenko, S.

M. Loktev, O. Soloviev, S. Savenko, and G. Vdovin, “Speckle imaging through turbulent atmosphere based on adaptable pupil segmentation,” Opt. Lett. 36(14), 2656–2658 (2011).
[Crossref] [PubMed]

Shabat, G.

L. Yaroslavsky, B. Fishbain, G. Shabat, and I. Ideses, “Superresolution in turbulent videos: making profit from damage,” Opt. Lett. 32(20), 3038–3040 (2007).
[Crossref] [PubMed]

Shah, M.

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 450–462 (2013).
[Crossref] [PubMed]

Shashua, E. H.

P. J. Burt and E. H. Shashua, “The laplacian pyramid as a compact image code,” IEEE Trans. Commun. 31(4), 532–540 (1983).
[Crossref]

Simske, S.

D. Li, R. M. Mersereau, and S. Simske, “Atmospheric turbulence-degraded image restoration using principal components analysis,” IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007).
[Crossref]

Soatto, S.

Y. Lou, S. H. Kang, S. Soatto, and A. L. Bertozzi, “Video stabilization of atmospheric turbulence distortion,” Inverse Probl. Imaging (Springfield) 7(3), 839–861 (2013).
[Crossref]

Sochen, N.

R. Gal, N. Kiryati, and N. Sochen, “Progress in the restoration of image sequences degraded by atmospheric turbulence,” Pattern Recognit. Lett. 48, 8–14 (2014).
[Crossref]

Soloviev, O.

M. Loktev, O. Soloviev, S. Savenko, and G. Vdovin, “Speckle imaging through turbulent atmosphere based on adaptable pupil segmentation,” Opt. Lett. 36(14), 2656–2658 (2011).
[Crossref] [PubMed]

Sommer, G.

M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49(12), 3136–3144 (2001).
[Crossref]

Vdovin, G.

M. Loktev, O. Soloviev, S. Savenko, and G. Vdovin, “Speckle imaging through turbulent atmosphere based on adaptable pupil segmentation,” Opt. Lett. 36(14), 2656–2658 (2011).
[Crossref] [PubMed]

Wadhwa, N.

N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Phase-based video motion processing,” ACM TOG 32(4), 80 (2013).
[Crossref]

Weiss, R.

E. Repasi and R. Weiss, “Computer Simulation of Image Degradations by Atmospheric Turbulence for Horizontal Views,” Proc. SPIE 8014, 80140U (2011).
[Crossref]

E. Repasi and R. Weiss, “Analysis of Image Distortions by Atmospheric Turbulence and Computer Simulation of Turbulence Effects,” Proc. SPIE 6941, 69410S (2008).
[Crossref]

Yaroslavsky, L.

L. Yaroslavsky, B. Fishbain, G. Shabat, and I. Ideses, “Superresolution in turbulent videos: making profit from damage,” Opt. Lett. 32(20), 3038–3040 (2007).
[Crossref] [PubMed]

Zhou, X.

P. Ruiz, X. Zhou, J. Mateos, R. Molina, and A. K. Katsaggelos, “Variational Bayesian Blind Image Deconvolution: A review,” Digit. Signal Process. 47, 116–127 (2015).
[Crossref]

Zhu, X.

X. Zhu and P. Milanfar, “Removing atmospheric Turbulence via Space-Invariant Deconvolution,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 157–170 (2013).
[Crossref] [PubMed]

X. Zhu and P. Milanfar, “Image reconstruction from videos distorted by atmospheric turbulence,” Proc. SPIE 7543(1), 423–426 (2010).

ACM TOG (1)

N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Phase-based video motion processing,” ACM TOG 32(4), 80 (2013).
[Crossref]

Digit. Signal Process. (1)

P. Ruiz, X. Zhou, J. Mateos, R. Molina, and A. K. Katsaggelos, “Variational Bayesian Blind Image Deconvolution: A review,” Digit. Signal Process. 47, 116–127 (2015).
[Crossref]

IEEE Geosci. Remote Sens. Lett. (1)

D. Li, R. M. Mersereau, and S. Simske, “Atmospheric turbulence-degraded image restoration using principal components analysis,” IEEE Geosci. Remote Sens. Lett. 4(3), 340–344 (2007).
[Crossref]

IEEE Trans. Commun. (1)

P. J. Burt and E. H. Shashua, “The laplacian pyramid as a compact image code,” IEEE Trans. Commun. 31(4), 532–540 (1983).
[Crossref]

IEEE Trans. Image Process. (1)

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. R. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22(6), 2398–2408 (2013).
[Crossref] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (2)

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 450–462 (2013).
[Crossref] [PubMed]

X. Zhu and P. Milanfar, “Removing atmospheric Turbulence via Space-Invariant Deconvolution,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 157–170 (2013).
[Crossref] [PubMed]

IEEE Trans. Signal Process. (1)

M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Trans. Signal Process. 49(12), 3136–3144 (2001).
[Crossref]

Inverse Probl. Imaging (Springfield) (1)

Y. Lou, S. H. Kang, S. Soatto, and A. L. Bertozzi, “Video stabilization of atmospheric turbulence distortion,” Inverse Probl. Imaging (Springfield) 7(3), 839–861 (2013).
[Crossref]

J. Computer Vis. Res. (1)

P. Kovesi, “Image features from phase congruency,” J. Computer Vis. Res. 1(3), 1–26 (1999).

J. Opt. Soc. Am. A (1)

I. Dror and N. S. Kopeika, “Experimental comparison of turbulence MTF and aerosol MTF through open atmosphere,” J. Opt. Soc. Am. A 12(5), 970–980 (1995).
[Crossref]

Opt. Express (1)

D. A. Hope, S. M. Jefferies, M. Hart, and J. G. Nagy, “High-resolution speckle imaging through strong atmospheric turbulence,” Opt. Express 24(11), 12116–12129 (2016).
[Crossref] [PubMed]

Opt. Lett. (2)

L. Yaroslavsky, B. Fishbain, G. Shabat, and I. Ideses, “Superresolution in turbulent videos: making profit from damage,” Opt. Lett. 32(20), 3038–3040 (2007).
[Crossref] [PubMed]

M. Loktev, O. Soloviev, S. Savenko, and G. Vdovin, “Speckle imaging through turbulent atmosphere based on adaptable pupil segmentation,” Opt. Lett. 36(14), 2656–2658 (2011).
[Crossref] [PubMed]

Pattern Recognit. Lett. (2)

R. Gal, N. Kiryati, and N. Sochen, “Progress in the restoration of image sequences degraded by atmospheric turbulence,” Pattern Recognit. Lett. 48, 8–14 (2014).
[Crossref]

M. C. Morrone and R. A. Owens, “Feature detection from local energy,” Pattern Recognit. Lett. 6(5), 303–313 (1987).
[Crossref]

Proc. SPIE (3)

E. Repasi and R. Weiss, “Analysis of Image Distortions by Atmospheric Turbulence and Computer Simulation of Turbulence Effects,” Proc. SPIE 6941, 69410S (2008).
[Crossref]

E. Repasi and R. Weiss, “Computer Simulation of Image Degradations by Atmospheric Turbulence for Horizontal Views,” Proc. SPIE 8014, 80140U (2011).
[Crossref]

X. Zhu and P. Milanfar, “Image reconstruction from videos distorted by atmospheric turbulence,” Proc. SPIE 7543(1), 423–426 (2010).

Other (7)

R. K. Tyson, Principles of Adaptive Optics (Academic, 1991).

Institute of Astronomy, University of Cambridge, “Lucky Imaging”, http://www.ast.cam.ac.uk/~optics/Lucky_Web_Site/index.htm

A. V. Oppenheim and J. S. Lim, “The importance of phase in signals,” InProceedings of the IEEE (IEEE, 1981), 69(5), pp, 529–541.

O. Haik and Y. Yitzhaky, http://www.ee.bgu.ac.il/~itzik/VideosOE06/

N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “SIGGRAPH 2013 Phase-Based Video Motion Processing”, http://people.csail.mit.edu/nwadhwa/phase-video/

F. J. Madrid-Cuevas, R. Medina-Carnicer, Á. Carmona-Poyato, and N. L. Fernández-García, “Dominant Points Detection Using Phase Congruence,” Iberian Conference on Pattern Recognition and Image Analysis (Springer Berlin Heidelberg, 2007), pp.138–145.
[Crossref]

N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Riesz pyramids for fast phase-based video magnification,” In proceedings of ICCP, (IEEE, 2014), pp.1–10.

Supplementary Material (8)

Name	Description
» Visualization 1: MP4 (1663 KB)	The effects of omega sub c
» Visualization 2: MP4 (3175 KB)	Simulated experiments
» Visualization 3: MP4 (7395 KB)	SNR 5db
» Visualization 4: MP4 (6916 KB)	SNR 15db
» Visualization 5: MP4 (6352 KB)	SNR 25db
» Visualization 6: MP4 (4747 KB)	Sequence1 stabilization results
» Visualization 7: MP4 (4527 KB)	Sequence 2 stabilization results
» Visualization 8: MP4 (3107 KB)	Sequence 3 stabilization results

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.

Click here to see a list of articles that cite this paper

Fig. 1 Block diagram for the proposed video stabilization framework.

Download Full Size | PPT Slide | PDF

Fig. 2 LRP and Local information (a) Input image, (b) LRP, (c) Local amplitude and local phase.

Download Full Size | PPT Slide | PDF

Fig. 3 The high-frequency fluctuations of turbulence distortion. (a) One frame of Sequence 2. (b) Local amplitude and local phase temporal varying of the green point .

Download Full Size | PPT Slide | PDF

Fig. 4 Latent sharp image used for simulation.

Download Full Size | PPT Slide | PDF

Fig. 5 The effects of ω_c (see Visualization 1) (a) one frame from the simulated video, (b) ω_c = 0.1, (c) ω_c = 0.3, (d) ω_c = 0.5, (e) ω_c = 0.8.

Download Full Size | PPT Slide | PDF

Fig. 6 Simulated comparison results (see Visualization 2) (a) [12] output, (b) [11] output, (c) the proposed output (with ω_c = 0.1).

Download Full Size | PPT Slide | PDF

Fig. 7 Results of additional noises (see Visualization 3, Visualization 4, Visualization 5) (a) one frame of simulated video with noise, (b) one frame of [11] output, (c) one frame of [12] output, (d) one frame of the proposed output.

Download Full Size | PPT Slide | PDF

Fig. 8 Sequence1 stabilization results (see Visualization 6) (a) one frame from sequence 1, (b) [12] output, (c) [11] output, (d) the proposed output.

Download Full Size | PPT Slide | PDF

Fig. 9 Sequence 2 stabilization results (see Visualization 7) (a) one frame from sequence 2, (b) [12] output, (c) [11] output, (d) the proposed output.

Download Full Size | PPT Slide | PDF

Fig. 10 Sequence 3 stabilization results (see Visualization 8) (a) one frame from sequence 3, (b) [12] output, (c) [11] output, (d) the proposed output.

Download Full Size | PPT Slide | PDF

Tables (4)

Table 1 Average PSNR (dB) for differentω_c values

View Table | View all tables in this article

Table 2 Comparison of the Average PSNR (dB)

View Table | View all tables in this article

Table 3 Performance with white Gaussian noise

View Table | View all tables in this article

Table 4 Comparison of running time (s)

View Table | View all tables in this article

Equations (14)

Equations on this page are rendered with MathJax. Learn more.

g_{k} (i, j) = R E D U C E (g_{k - 1}) = \sum_{m = - 2}^{2} \sum_{n = - 2}^{2} w (m, n) g_{k - 1} (2 i + m, 2 j + n)

g_{k, ↑ 2} (i, j) = E X P A N D (g_{k}) = 4 \sum_{m = - 2}^{2} \sum_{n = - 2}^{2} w (m, n) g_{k} (\frac{i - m}{2}, \frac{j - n}{2})

L^{{1}} = I - E X P A N D (g_{2}) = I - g_{2, ↑ 2}

L^{{k}} = g_{k} - E X P A N D (g_{k + 1}) = g_{k} - g_{k + 1, ↑ 2} \begin{array}{l}  \end{array} f o r \begin{array}{l}  \end{array} 2 \leq k < N

L^{{N}} = g_{N}

g_{k - 1} = L^{{k - 1}} + E X P A N D (g_{k})

I_{o u t} = L^{{1}} + E X P A N D (g_{2})

ℜ (x) = (\begin{matrix} R_{1} (x) \\ R_{2} (x) \end{matrix}) = (\begin{matrix} h_{x} * I (x) \\ h_{y} * I (x) \end{matrix})

I = A \cos (φ), R_{1} = A \sin (φ) \cos (θ), R_{2} = A \sin (φ) \sin (θ)

A = \sqrt{I^{2} + R_{1}^{2} + R_{2}^{2}}

φ = \cos^{- 1} (I / A)

I_{T u r b u} (x, y, t) = [A (x, y) + Δ (x, y, t)] \cos [φ (x, y) + δ (x, y, t)]

{‖ H_{B u t t e r} (ω) ‖}^{2} = \frac{1}{1 + {(ω / ω_{c})}^{2}}

\sum_{k = 0}^{N} a [k] \cdot o u t [n - k] = \sum_{k = 0}^{M} b [k] \cdot i n p u t [n - k]

Video	Original	[11]	[12]	Proposed
Simulated	15.6766	10.4913	16.9355	17.4870
Sequence 1	33.2185	38.3875	39.0879	41.8363
Sequence 2	25.0087	33.4140	30.3715	33.8323
Sequence 3	37.7525	41.5464	41.3036	41.5938

Noise SNR (dB)	5	15	25
Original PSNR	7.8847	12.7550	14.4854
[11] PSNR	5.7334	7.6738	9.7326
[12] PSNR	11.7968	15.1002	15.7375
Proposed PSNR	12.6829	15.5373	16.1482

Video	Resolution (w × h × frame)	[11]	[12]	Proposed
Simulated	280 × 280 × 100	11.22	24.85	2.66
Sequence 1	1280 × 720 × 101	206.58	359.64	36.13
Sequence 2	636 × 480 × 85	54.78	103.36	9.49
Sequence 3	636 × 480 × 85	56.54	99.77	9.74

Video	Original	[11]	[12]	Proposed
Simulated	15.6766	10.4913	16.9355	17.4870
Sequence 1	33.2185	38.3875	39.0879	41.8363
Sequence 2	25.0087	33.4140	30.3715	33.8323
Sequence 3	37.7525	41.5464	41.3036	41.5938

Noise SNR (dB)	5	15	25
Original PSNR	7.8847	12.7550	14.4854
[11] PSNR	5.7334	7.6738	9.7326
[12] PSNR	11.7968	15.1002	15.7375
Proposed PSNR	12.6829	15.5373	16.1482

Abstract

References

2016 (1)

2015 (1)

2014 (1)

2013 (5)

2011 (2)

2010 (1)

2008 (1)

2007 (2)

2001 (1)

1999 (1)

1995 (1)

1987 (1)

1983 (1)

Achim, A.

Anantrasirichai, N.

Bertozzi, A. L.

Bull, D. R.

Burt, P. J.

Dror, I.

Durand, F.

Felsberg, M.

Fishbain, B.

Freeman, W. T.

Gal, R.

Hart, M.

Hope, D. A.

Ideses, I.

Jefferies, S. M.

Kang, S. H.

Katsaggelos, A. K.

Kingsbury, N. G.

Kiryati, N.

Kopeika, N. S.

Kovesi, P.

Li, D.

Li, X.

Lim, J. S.

Loktev, M.

Lou, Y.

Mateos, J.

Mersereau, R. M.

Milanfar, P.

Molina, R.

Morrone, M. C.

Nagy, J. G.

Oppenheim, A. V.

Oreifej, O.

Owens, R. A.

Repasi, E.

Rubinstein, M.

Ruiz, P.

Savenko, S.

Shabat, G.

Shah, M.

Shashua, E. H.

Simske, S.

Soatto, S.

Sochen, N.

Soloviev, O.

Sommer, G.

Vdovin, G.

Wadhwa, N.

Weiss, R.

Yaroslavsky, L.

Zhou, X.

Zhu, X.

ACM TOG (1)

Digit. Signal Process. (1)

IEEE Geosci. Remote Sens. Lett. (1)

IEEE Trans. Commun. (1)

IEEE Trans. Image Process. (1)

IEEE Trans. Pattern Anal. Mach. Intell. (2)

IEEE Trans. Signal Process. (1)

Inverse Probl. Imaging (Springfield) (1)

J. Computer Vis. Res. (1)

J. Opt. Soc. Am. A (1)

Opt. Express (1)

Opt. Lett. (2)