Abstract

Long-distance surveillance is a challenging task because of atmospheric turbulence that causes time-varying image shifts and blurs in images. These distortions become more significant as the imaging distance increases. This paper presents a new method for compensating image shifting in a video sequence while keeping real moving objects in the video unharmed. In this approach, firstly, a highly accurate and fast optical flow technique is applied to estimate the motion vector maps of the input frames and a centroid algorithm is employed to generate a geometrically correct frame in which there is no moving object. The second step involves applying an algorithm for detecting real moving objects in the video sequence and then restoring it with those objects unaffected. The performance of the proposed method is verified by comparing it with that of a state-of-the-art approach. Simulation experiments using both synthetic and real-life surveillance videos demonstrate that this method significantly improves the accuracy of image restoration while preserving moving objects.

© 2015 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Simple and efficient approach for restoration of non-uniformly warped images

Kalyan Kumar Halder, Murat Tahtali, and Sreenatha G. Anavatti
Appl. Opt. 53(25) 5576-5584 (2014)

Correction of geometrically distorted underwater images using shift map analysis

Kalyan Kumar Halder, Manoranjan Paul, Murat Tahtali, Sreenatha G. Anavatti, and Manzur Murshed
J. Opt. Soc. Am. A 34(4) 666-673 (2017)

Detecting and tracking moving objects in long-distance imaging through turbulent medium

Eli Chen, Oren Haik, and Yitzhak Yitzhaky
Appl. Opt. 53(6) 1181-1190 (2014)

References

  • View by:
  • |
  • |
  • |

  1. R. K. Tyson, Introduction to Adaptive Optics (SPIE, 2000).
    [Crossref]
  2. K. K. Halder, M. Tahtali, and S. G. Anavatti, “Simple and efficient approach for restoration of non-uniformly warped images,” Appl. Opt. 53, 5576–5584 (2014).
    [Crossref] [PubMed]
  3. D. H. Frakes, J. W. Monaco, and M. J. T. Smith, “Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1881–1884.
  4. N. Lu, J. Wang, Q. H. Wu, and L. Yang, “An improved motion detection method for real-time surveillance,” IAENG Int,”. J. Comput. Sci..  35, 119 (2008).
  5. A. S. Deshmukh, S. S. Medasani, and G. R. Reddy, “Moving object detection from images distorted by atmospheric turbulence,” in Proceedings of the International Conference on Intelligent Systems and Signal Processing (Institute of Electrical and Electronics Engineers, 2013), pp. 122–127.
  6. S. Gepshtein, A. Shtainman, B. Fishbain, and L. P. Yaroslavsky, “Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,” in Proceedings of the European Signal Processing Conference (European Association for Signal Processing, 2004), pp. 477–480.
  7. L. P. Yaroslavsky, B. Fishbain, A. Shteinman, and S. Gepshtein, “Processing and fusion of thermal and video sequences for terrestrial long range observation systems,” in Proceedings of the International Conference on Information Fusion (International Society of Information Fusion, 2004), pp. 848–855.
  8. B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Real-time stabilization of long range observation system turbulent video,” J. Real-Time Image Proc. 2, 11–22 (2007).
    [Crossref]
  9. B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Spatial, temporal, and interchannel image data fusion for long-distance terrestrial observation systems,” Adv. Opt. Technol. 2008, 546808 (2008).
    [Crossref]
  10. J. Lou, H. Yang, W. Hu, and T. Tan, “An illumination invariant change detection algorithm,” in Proceedings of the Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies, 2002), pp. 1–6.
  11. N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22, 2398–2408 (2013).
    [Crossref] [PubMed]
  12. O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 450–462 (2013).
    [Crossref]
  13. T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory for warping,” Lect. Notes Comput. Sci. 3024, 25–36 (2004).
    [Crossref]
  14. C. Liu, “Beyond pixels: exploring new representations and applications for motion analysis,” Ph.D. dissertation (Massachusetts Institute of Technology, 2009).
  15. K. K. Halder, M. Tahtali, and S. G. Anavatti, “Model-free prediction of atmospheric warp based on artificial neural network,” Appl. Opt. 53, 7087–7094 (2014).
    [Crossref] [PubMed]
  16. M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
    [Crossref]
  17. MathWorks, “Motion-based multiple object tracking,” http://www.mathworks.com .
  18. N. Goyette, P. -M. Jodoin, F. Porikli, J. Konrad, and P. Ishwar, “changedetection.net: A new change detection benchmark dataset,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (Institute of Electrical and Electronics Engineers, 2012), pp. 1–8.

2014 (2)

2013 (2)

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22, 2398–2408 (2013).
[Crossref] [PubMed]

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 450–462 (2013).
[Crossref]

2008 (2)

N. Lu, J. Wang, Q. H. Wu, and L. Yang, “An improved motion detection method for real-time surveillance,” IAENG Int,”. J. Comput. Sci..  35, 119 (2008).

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Spatial, temporal, and interchannel image data fusion for long-distance terrestrial observation systems,” Adv. Opt. Technol. 2008, 546808 (2008).
[Crossref]

2007 (1)

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Real-time stabilization of long range observation system turbulent video,” J. Real-Time Image Proc. 2, 11–22 (2007).
[Crossref]

2006 (1)

M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
[Crossref]

2004 (1)

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory for warping,” Lect. Notes Comput. Sci. 3024, 25–36 (2004).
[Crossref]

Achim, A.

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22, 2398–2408 (2013).
[Crossref] [PubMed]

Anantrasirichai, N.

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22, 2398–2408 (2013).
[Crossref] [PubMed]

Anavatti, S. G.

Brox, T.

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory for warping,” Lect. Notes Comput. Sci. 3024, 25–36 (2004).
[Crossref]

Bruhn, A.

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory for warping,” Lect. Notes Comput. Sci. 3024, 25–36 (2004).
[Crossref]

Bull, D.

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22, 2398–2408 (2013).
[Crossref] [PubMed]

Deshmukh, A. S.

A. S. Deshmukh, S. S. Medasani, and G. R. Reddy, “Moving object detection from images distorted by atmospheric turbulence,” in Proceedings of the International Conference on Intelligent Systems and Signal Processing (Institute of Electrical and Electronics Engineers, 2013), pp. 122–127.

Fishbain, B.

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Spatial, temporal, and interchannel image data fusion for long-distance terrestrial observation systems,” Adv. Opt. Technol. 2008, 546808 (2008).
[Crossref]

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Real-time stabilization of long range observation system turbulent video,” J. Real-Time Image Proc. 2, 11–22 (2007).
[Crossref]

L. P. Yaroslavsky, B. Fishbain, A. Shteinman, and S. Gepshtein, “Processing and fusion of thermal and video sequences for terrestrial long range observation systems,” in Proceedings of the International Conference on Information Fusion (International Society of Information Fusion, 2004), pp. 848–855.

S. Gepshtein, A. Shtainman, B. Fishbain, and L. P. Yaroslavsky, “Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,” in Proceedings of the European Signal Processing Conference (European Association for Signal Processing, 2004), pp. 477–480.

Frakes, D. H.

D. H. Frakes, J. W. Monaco, and M. J. T. Smith, “Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1881–1884.

Fraser, D.

M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
[Crossref]

Gepshtein, S.

S. Gepshtein, A. Shtainman, B. Fishbain, and L. P. Yaroslavsky, “Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,” in Proceedings of the European Signal Processing Conference (European Association for Signal Processing, 2004), pp. 477–480.

L. P. Yaroslavsky, B. Fishbain, A. Shteinman, and S. Gepshtein, “Processing and fusion of thermal and video sequences for terrestrial long range observation systems,” in Proceedings of the International Conference on Information Fusion (International Society of Information Fusion, 2004), pp. 848–855.

Goyette, N.

N. Goyette, P. -M. Jodoin, F. Porikli, J. Konrad, and P. Ishwar, “changedetection.net: A new change detection benchmark dataset,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (Institute of Electrical and Electronics Engineers, 2012), pp. 1–8.

Halder, K. K.

Hu, W.

J. Lou, H. Yang, W. Hu, and T. Tan, “An illumination invariant change detection algorithm,” in Proceedings of the Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies, 2002), pp. 1–6.

Ideses, I. A.

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Spatial, temporal, and interchannel image data fusion for long-distance terrestrial observation systems,” Adv. Opt. Technol. 2008, 546808 (2008).
[Crossref]

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Real-time stabilization of long range observation system turbulent video,” J. Real-Time Image Proc. 2, 11–22 (2007).
[Crossref]

Ishwar, P.

N. Goyette, P. -M. Jodoin, F. Porikli, J. Konrad, and P. Ishwar, “changedetection.net: A new change detection benchmark dataset,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (Institute of Electrical and Electronics Engineers, 2012), pp. 1–8.

Jodoin, P. -M.

N. Goyette, P. -M. Jodoin, F. Porikli, J. Konrad, and P. Ishwar, “changedetection.net: A new change detection benchmark dataset,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (Institute of Electrical and Electronics Engineers, 2012), pp. 1–8.

Kingsbury, N. G.

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22, 2398–2408 (2013).
[Crossref] [PubMed]

Konrad, J.

N. Goyette, P. -M. Jodoin, F. Porikli, J. Konrad, and P. Ishwar, “changedetection.net: A new change detection benchmark dataset,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (Institute of Electrical and Electronics Engineers, 2012), pp. 1–8.

Lambert, A. J.

M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
[Crossref]

Li, X.

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 450–462 (2013).
[Crossref]

Liu, C.

C. Liu, “Beyond pixels: exploring new representations and applications for motion analysis,” Ph.D. dissertation (Massachusetts Institute of Technology, 2009).

Lou, J.

J. Lou, H. Yang, W. Hu, and T. Tan, “An illumination invariant change detection algorithm,” in Proceedings of the Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies, 2002), pp. 1–6.

Lu, N.

N. Lu, J. Wang, Q. H. Wu, and L. Yang, “An improved motion detection method for real-time surveillance,” IAENG Int,”. J. Comput. Sci..  35, 119 (2008).

Medasani, S. S.

A. S. Deshmukh, S. S. Medasani, and G. R. Reddy, “Moving object detection from images distorted by atmospheric turbulence,” in Proceedings of the International Conference on Intelligent Systems and Signal Processing (Institute of Electrical and Electronics Engineers, 2013), pp. 122–127.

Monaco, J. W.

D. H. Frakes, J. W. Monaco, and M. J. T. Smith, “Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1881–1884.

Oreifej, O.

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 450–462 (2013).
[Crossref]

Papenberg, N.

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory for warping,” Lect. Notes Comput. Sci. 3024, 25–36 (2004).
[Crossref]

Porikli, F.

N. Goyette, P. -M. Jodoin, F. Porikli, J. Konrad, and P. Ishwar, “changedetection.net: A new change detection benchmark dataset,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (Institute of Electrical and Electronics Engineers, 2012), pp. 1–8.

Reddy, G. R.

A. S. Deshmukh, S. S. Medasani, and G. R. Reddy, “Moving object detection from images distorted by atmospheric turbulence,” in Proceedings of the International Conference on Intelligent Systems and Signal Processing (Institute of Electrical and Electronics Engineers, 2013), pp. 122–127.

Shah, M.

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 450–462 (2013).
[Crossref]

Shtainman, A.

S. Gepshtein, A. Shtainman, B. Fishbain, and L. P. Yaroslavsky, “Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,” in Proceedings of the European Signal Processing Conference (European Association for Signal Processing, 2004), pp. 477–480.

Shteinman, A.

L. P. Yaroslavsky, B. Fishbain, A. Shteinman, and S. Gepshtein, “Processing and fusion of thermal and video sequences for terrestrial long range observation systems,” in Proceedings of the International Conference on Information Fusion (International Society of Information Fusion, 2004), pp. 848–855.

Smith, M. J. T.

D. H. Frakes, J. W. Monaco, and M. J. T. Smith, “Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1881–1884.

Tahtali, M.

Tan, T.

J. Lou, H. Yang, W. Hu, and T. Tan, “An illumination invariant change detection algorithm,” in Proceedings of the Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies, 2002), pp. 1–6.

Tyson, R. K.

R. K. Tyson, Introduction to Adaptive Optics (SPIE, 2000).
[Crossref]

Wang, J.

N. Lu, J. Wang, Q. H. Wu, and L. Yang, “An improved motion detection method for real-time surveillance,” IAENG Int,”. J. Comput. Sci..  35, 119 (2008).

Weickert, J.

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory for warping,” Lect. Notes Comput. Sci. 3024, 25–36 (2004).
[Crossref]

Wu, Q. H.

N. Lu, J. Wang, Q. H. Wu, and L. Yang, “An improved motion detection method for real-time surveillance,” IAENG Int,”. J. Comput. Sci..  35, 119 (2008).

Yang, H.

J. Lou, H. Yang, W. Hu, and T. Tan, “An illumination invariant change detection algorithm,” in Proceedings of the Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies, 2002), pp. 1–6.

Yang, L.

N. Lu, J. Wang, Q. H. Wu, and L. Yang, “An improved motion detection method for real-time surveillance,” IAENG Int,”. J. Comput. Sci..  35, 119 (2008).

Yaroslavsky, L. P.

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Spatial, temporal, and interchannel image data fusion for long-distance terrestrial observation systems,” Adv. Opt. Technol. 2008, 546808 (2008).
[Crossref]

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Real-time stabilization of long range observation system turbulent video,” J. Real-Time Image Proc. 2, 11–22 (2007).
[Crossref]

S. Gepshtein, A. Shtainman, B. Fishbain, and L. P. Yaroslavsky, “Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,” in Proceedings of the European Signal Processing Conference (European Association for Signal Processing, 2004), pp. 477–480.

L. P. Yaroslavsky, B. Fishbain, A. Shteinman, and S. Gepshtein, “Processing and fusion of thermal and video sequences for terrestrial long range observation systems,” in Proceedings of the International Conference on Information Fusion (International Society of Information Fusion, 2004), pp. 848–855.

Adv. Opt. Technol. (1)

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Spatial, temporal, and interchannel image data fusion for long-distance terrestrial observation systems,” Adv. Opt. Technol. 2008, 546808 (2008).
[Crossref]

Appl. Opt. (2)

IEEE Trans. Image Process. (1)

N. Anantrasirichai, A. Achim, N. G. Kingsbury, and D. Bull, “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Trans. Image Process. 22, 2398–2408 (2013).
[Crossref] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” IEEE Trans. Pattern Anal. Mach. Intell. 35, 450–462 (2013).
[Crossref]

J. Comput. Sci. (1)

N. Lu, J. Wang, Q. H. Wu, and L. Yang, “An improved motion detection method for real-time surveillance,” IAENG Int,”. J. Comput. Sci..  35, 119 (2008).

J. Real-Time Image Proc. (1)

B. Fishbain, L. P. Yaroslavsky, and I. A. Ideses, “Real-time stabilization of long range observation system turbulent video,” J. Real-Time Image Proc. 2, 11–22 (2007).
[Crossref]

Lect. Notes Comput. Sci. (1)

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory for warping,” Lect. Notes Comput. Sci. 3024, 25–36 (2004).
[Crossref]

Proc. SPIE (1)

M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
[Crossref]

Other (9)

MathWorks, “Motion-based multiple object tracking,” http://www.mathworks.com .

N. Goyette, P. -M. Jodoin, F. Porikli, J. Konrad, and P. Ishwar, “changedetection.net: A new change detection benchmark dataset,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (Institute of Electrical and Electronics Engineers, 2012), pp. 1–8.

C. Liu, “Beyond pixels: exploring new representations and applications for motion analysis,” Ph.D. dissertation (Massachusetts Institute of Technology, 2009).

R. K. Tyson, Introduction to Adaptive Optics (SPIE, 2000).
[Crossref]

D. H. Frakes, J. W. Monaco, and M. J. T. Smith, “Suppression of atmospheric turbulence in video using an adaptive control grid interpolation approach,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Institute of Electrical and Electronics Engineers, 2001), pp. 1881–1884.

A. S. Deshmukh, S. S. Medasani, and G. R. Reddy, “Moving object detection from images distorted by atmospheric turbulence,” in Proceedings of the International Conference on Intelligent Systems and Signal Processing (Institute of Electrical and Electronics Engineers, 2013), pp. 122–127.

S. Gepshtein, A. Shtainman, B. Fishbain, and L. P. Yaroslavsky, “Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration,” in Proceedings of the European Signal Processing Conference (European Association for Signal Processing, 2004), pp. 477–480.

L. P. Yaroslavsky, B. Fishbain, A. Shteinman, and S. Gepshtein, “Processing and fusion of thermal and video sequences for terrestrial long range observation systems,” in Proceedings of the International Conference on Information Fusion (International Society of Information Fusion, 2004), pp. 848–855.

J. Lou, H. Yang, W. Hu, and T. Tan, “An illumination invariant change detection algorithm,” in Proceedings of the Asian Conference on Computer Vision (Asian Federation of Computer Vision Societies, 2002), pp. 1–6.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1 Motion vector maps of the non-uniformly warped frame (a) with respect to the reference frame (b) are shown in (c) and (d) for the x- and y- directional components, respectively.
Fig. 2
Fig. 2 (a) Sample warped frame, (b) geometrically stable frame, (c) difference image between (a) and (b), (d) mask MD generated from the difference image, (e) magnitude map of the motion vector between (a) and (b), and (f) mask MM generated from the motion vector map overlaid with the mask MD.
Fig. 3
Fig. 3 Results from image restoration for the synthetic sequence: (a) sample of geometrically distorted frames, (b) masks generated using the Fishbain method, (c) frames restored using the Fishbain method, (d) masks (MC) generated using the proposed method, and (e) frames restored using the proposed method.
Fig. 4
Fig. 4 NMSE plots between two successive frames of the warped and restored sequences using the Fishbain and proposed methods for the synthetic sequence.
Fig. 5
Fig. 5 Results from image restoration for the real-life sequence: (a) sample of geometrically distorted frames, (b) masks generated using the Fishbain method, (c) frames restored using the Fishbain method, (d) masks (MC) generated using the proposed method, and (e) frames restored using the proposed method.
Fig. 6
Fig. 6 NMSE plots between two successive frames of the warped and restored sequences using the Fishbain and proposed methods for the real-life sequence.

Tables (1)

Tables Icon

Table 1 Performance comparison of the methods with the synthetic sequence.

Equations (17)

Equations on this page are rendered with MathJax. Learn more.

F ( x , y , t ) = F ( x + s x , y + s y , t + 1 ) .
E ( s x , s y ) = ψ ( | F ( p + s ) F ( p ) | 2 ) + α ϕ ( | s x | 2 + | s y | 2 ) d p .
E ( d s x , d s y ) = ψ ( | F ( p + s + d s ) F ( p ) | 2 ) + α ϕ ( | ( s x + d s x ) | 2 + | ( s y + d s y ) | 2 ) d p .
F ( p + s + d s ) F ( p ) F z ( p ) + F x ( p ) d s x + F y ( p ) d s y .
E ( d S X , d S Y ) = p ψ ( ( δ p T ( F z + F x d S X + F y d S Y ) ) 2 ) + α ϕ ( ( δ p T D x ( S X + d S X ) ) 2 + ( δ p T D y ( S X + d S X ) ) 2 + ( δ p T D x ( S Y + d S Y ) ) 2 + ( δ p T D y ( S Y + d S Y ) ) 2 ) .
E ( d S X , d S Y ) = p ψ ( f p ) + α ϕ ( g p ) .
r x ( p ) = c x ( p ) + s x ( p + c ) , r y ( p ) = c y ( p ) + s y ( p + c ) .
F ( p ) = F ( p + r ) ,
D ( p ) = | F ( p ) F ( p ) ¯ | .
T D L = G D σ D + K D L , T D H = T D L + K D H ,
M D ( p ) = { 0 if D ( p ) < T D L 1 if D ( p ) > T D H D ( p ) T D H otherwise .
R ( p ) = r x 2 ( p ) + r y 2 ( p )
T M = G M σ M + K M ,
M M ( p ) = { 0 if R ( p ) < T M 1 otherwise .
M C ( p ) = M D ( p ) · M M ( p ) ,
F R ( p ) = F ( p ) ¯ · ( 1 M C ( p ) ) + F ( p ) · M C ( p ) ,
NMSE = ( F R F G ) 2 ¯ F ¯ R F ¯ G ,

Metrics