Abstract

Imaging through a fluctuating air-water surface is a challenging task since light rays bent by unknown amounts lead to complex geometric distortions. This paper presents a new algorithm to undistort dynamic refractive effects. An iterative robust registration algorithm is employed to overcome the structural turbulence of the waves of the frames by registering each frame to a reference frame. To get the high-quality reference frame, the image is reconstructed by the patches selected from the sequence frames. A blind deconvolution algorithm is also performed to improve the reference frame. Experiments show our method exhibits significant improvement over other methods.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Correction of geometrically distorted underwater images using shift map analysis

Kalyan Kumar Halder, Manoranjan Paul, Murat Tahtali, Sreenatha G. Anavatti, and Manzur Murshed
J. Opt. Soc. Am. A 34(4) 666-673 (2017)

Simple and efficient approach for restoration of non-uniformly warped images

Kalyan Kumar Halder, Murat Tahtali, and Sreenatha G. Anavatti
Appl. Opt. 53(25) 5576-5584 (2014)

Restoring atmospheric-turbulence-degraded images

Md. Hasan Furhad, Murat Tahtali, and Andrew Lambert
Appl. Opt. 55(19) 5082-5090 (2016)

References

  • View by:
  • |
  • |
  • |

  1. A. Efros, V. Isler, J. Shi, and M. Visontai, “Seeing through water,” in Proceedings of Neural Information Processing Systems (Neural Information Processing Systems Foundation, 2004), pp. 393–400.
  2. A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by water waves,” in Proceedings of International Conference on Pattern Recognition (IEEE, 2006), pp. 421–424.
  3. Z. Wen, A. Lambert, D. Fraser, and H. Li, “Bispectral analysis and recovery of images distorted by a moving water surface,” Appl. Opt. 49(33), 6376–6384 (2010).
    [Crossref] [PubMed]
  4. Y. Tian and S. G. Narasimhan, “Seeing through water: image restoration using model-based tracking,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 2303–2310.
    [Crossref]
  5. O. Oreifej, G. Shu, T. Pace, and M. Shah, “A two-stage reconstruction approach for seeing through water,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 1153–1160.
    [Crossref]
  6. K. K. Halder, M. Paul, M. Tahtali, S. G. Anavatti, and M. Murshed, “Correction of geometrically distorted underwater images using shift map analysis,” J. Opt. Soc. Am. A 34(4), 666–673 (2017).
    [Crossref] [PubMed]
  7. A. V. Kanaev, W. Hou, S. R. Restaino, S. Matt, and S. Gładysz, “Restoration of images degraded by underwater turbulence using structure tensor oriented image quality (STOIQ) metric,” Opt. Express 23(13), 17077–17090 (2015).
    [Crossref] [PubMed]
  8. L. Li, Q. Wang, and Z. Xiao, “Underwater Image Restoration Algorithm from Distorted Video,” J. Syst. Simul. 24(1), 188–196 (2012).
  9. R. Zhang, D. He, Y. Li, L. Huang, and X. Bao, “Synthetic imaging through wavy water surface with centroid evolution,” Opt. Express 26(20), 26009–26019 (2018).
    [Crossref] [PubMed]
  10. C. Cox and W. Munk, “Slopes of the sea surface deduced from photographs of sun glitter,” Bull. Scripps Inst. Oceanogr. 6, 401–479 (1956).
  11. Z. Wen and D. Fraser, A. lambert and H. Li, “Reconstruction of Underwater Image by Bispectrum,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. 545–548.
  12. Z. Wen, A. Lambert, and D. Fraser, “Reconstruction of Imagery Reflected from Water Surface,” inSignal Recovery and Synthesis (OSA, 2007), pp. 18–20.
  13. N. Paul, A. de Chillaz, and J.-L. Collette, “On-line restoration for turbulence degraded video in nuclear power plant reactors,” Signal Image Video Process. 9(3), 601–610 (2015).
    [Crossref]
  14. H. Murase, “Surface shape reconstruction of a nonrigid transparent object using refraction and motion,” IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 1045–1052 (1992).
    [Crossref]
  15. D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
    [Crossref]
  16. T. Treibitz, Y. Y. Schechner, and H. Singh, “Flat refractive geometry,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008).
  17. D. G. Turlaev and L. S. Dolin, “On Observing Underwater Objects through a Wavy Water Surface: A New Algorithm for Image Correction and Laboratory Experiment,” Izv., Atmos. Ocean. Phys. 49(3), 339–345 (2013).
    [Crossref]
  18. Y. Tian and S. G. Narasimhan, “Globally Optimal Estimation of Nonrigid Image Distortion,” Int. J. Comput. Vis. 98(3), 279–302 (2012).
    [Crossref]
  19. Y. Tian and S. G. Narasimhan, “Theory and Practice of Hierarchical Data-driven Descent for Optimal Deformation Estimation,” Int. J. Comput. Vis. 115(1), 44–67 (2015).
    [Crossref]
  20. K. Seemakurthy and A. N. Rajagopalan, “Deskewing of Underwater Images,” IEEE Trans. Image Process. 24(3), 1046–1059 (2015).
    [Crossref] [PubMed]
  21. K. K. Halder, M. Tahtali, and S. G. Anavatti, “Simple and efficient approach for restoration of non-uniformly warped images,” Appl. Opt. 53(25), 5576–5584 (2014).
    [Crossref] [PubMed]
  22. K. K. Halder, M. Tahtali, and S. G. Anavatti, “An Artificial Neural Network Approach for Underwater Warp Prediction,” in Proceedings of Hellenic Conference on Artificial Intelligence (ACM, 2014), pp. 384–394.
    [Crossref]
  23. A. V. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “Imaging Through the Air-Water Interface,” in Proceedings of Computational Optical Sensing and Imaging (OSA, 2009), pp. 13–15.
  24. A. V. Kanaev, W. Hou, S. Woods, and L. N. Smith, “Restoration of turbulence degraded underwater images,” Opt. Eng. 51(5), 057007 (2012).
    [Crossref]
  25. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
    [Crossref] [PubMed]
  26. D. S. C. Biggs and M. Andrews, “Acceleration of iterative image restoration algorithms,” Appl. Opt. 36(8), 1766–1775 (1997).
    [Crossref] [PubMed]
  27. D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Med. Imaging 18(8), 712–721 (1999).
    [Crossref] [PubMed]
  28. S. Lee, G. Wolberg, and S. Y. Shin, “Scattered data interpolation with multilevel B-splines,” IEEE Trans. Vis. Comput. Graph. 3(3), 228–244 (1997).
    [Crossref]
  29. J. V. Hajnal, N. Saeed, E. J. Soar, A. Oatridge, I. R. Young, and G. M. Bydder, “A Registration and Interpolation Procedure for Subvoxel Matching of Serially Acquired MR Images,” J. Comput. Assist. Tomogr. 19(2), 289–296 (1995).
    [Crossref] [PubMed]
  30. S. Gabarda and G. Cristóbal, “Blind image quality assessment through anisotropy,” J. Opt. Soc. Am. A 24(12), B42–B51 (2007).
    [Crossref] [PubMed]

2018 (1)

2017 (1)

2015 (4)

N. Paul, A. de Chillaz, and J.-L. Collette, “On-line restoration for turbulence degraded video in nuclear power plant reactors,” Signal Image Video Process. 9(3), 601–610 (2015).
[Crossref]

Y. Tian and S. G. Narasimhan, “Theory and Practice of Hierarchical Data-driven Descent for Optimal Deformation Estimation,” Int. J. Comput. Vis. 115(1), 44–67 (2015).
[Crossref]

K. Seemakurthy and A. N. Rajagopalan, “Deskewing of Underwater Images,” IEEE Trans. Image Process. 24(3), 1046–1059 (2015).
[Crossref] [PubMed]

A. V. Kanaev, W. Hou, S. R. Restaino, S. Matt, and S. Gładysz, “Restoration of images degraded by underwater turbulence using structure tensor oriented image quality (STOIQ) metric,” Opt. Express 23(13), 17077–17090 (2015).
[Crossref] [PubMed]

2014 (1)

2013 (1)

D. G. Turlaev and L. S. Dolin, “On Observing Underwater Objects through a Wavy Water Surface: A New Algorithm for Image Correction and Laboratory Experiment,” Izv., Atmos. Ocean. Phys. 49(3), 339–345 (2013).
[Crossref]

2012 (3)

Y. Tian and S. G. Narasimhan, “Globally Optimal Estimation of Nonrigid Image Distortion,” Int. J. Comput. Vis. 98(3), 279–302 (2012).
[Crossref]

A. V. Kanaev, W. Hou, S. Woods, and L. N. Smith, “Restoration of turbulence degraded underwater images,” Opt. Eng. 51(5), 057007 (2012).
[Crossref]

L. Li, Q. Wang, and Z. Xiao, “Underwater Image Restoration Algorithm from Distorted Video,” J. Syst. Simul. 24(1), 188–196 (2012).

2010 (1)

2007 (1)

2006 (1)

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

2004 (1)

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

1999 (1)

D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Med. Imaging 18(8), 712–721 (1999).
[Crossref] [PubMed]

1997 (2)

S. Lee, G. Wolberg, and S. Y. Shin, “Scattered data interpolation with multilevel B-splines,” IEEE Trans. Vis. Comput. Graph. 3(3), 228–244 (1997).
[Crossref]

D. S. C. Biggs and M. Andrews, “Acceleration of iterative image restoration algorithms,” Appl. Opt. 36(8), 1766–1775 (1997).
[Crossref] [PubMed]

1995 (1)

J. V. Hajnal, N. Saeed, E. J. Soar, A. Oatridge, I. R. Young, and G. M. Bydder, “A Registration and Interpolation Procedure for Subvoxel Matching of Serially Acquired MR Images,” J. Comput. Assist. Tomogr. 19(2), 289–296 (1995).
[Crossref] [PubMed]

1992 (1)

H. Murase, “Surface shape reconstruction of a nonrigid transparent object using refraction and motion,” IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 1045–1052 (1992).
[Crossref]

1956 (1)

C. Cox and W. Munk, “Slopes of the sea surface deduced from photographs of sun glitter,” Bull. Scripps Inst. Oceanogr. 6, 401–479 (1956).

Ackerman, J.

A. V. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “Imaging Through the Air-Water Interface,” in Proceedings of Computational Optical Sensing and Imaging (OSA, 2009), pp. 13–15.

Anavatti, S. G.

Andrews, M.

Bao, X.

Biggs, D. S. C.

Bovik, A. C.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

Bydder, G. M.

J. V. Hajnal, N. Saeed, E. J. Soar, A. Oatridge, I. R. Young, and G. M. Bydder, “A Registration and Interpolation Procedure for Subvoxel Matching of Serially Acquired MR Images,” J. Comput. Assist. Tomogr. 19(2), 289–296 (1995).
[Crossref] [PubMed]

Carter, P. W.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Collette, J.-L.

N. Paul, A. de Chillaz, and J.-L. Collette, “On-line restoration for turbulence degraded video in nuclear power plant reactors,” Signal Image Video Process. 9(3), 601–610 (2015).
[Crossref]

Cox, C.

C. Cox and W. Munk, “Slopes of the sea surface deduced from photographs of sun glitter,” Bull. Scripps Inst. Oceanogr. 6, 401–479 (1956).

Cristóbal, G.

Dahme, G.

A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by water waves,” in Proceedings of International Conference on Pattern Recognition (IEEE, 2006), pp. 421–424.

de Chillaz, A.

N. Paul, A. de Chillaz, and J.-L. Collette, “On-line restoration for turbulence degraded video in nuclear power plant reactors,” Signal Image Video Process. 9(3), 601–610 (2015).
[Crossref]

Dolin, L. S.

D. G. Turlaev and L. S. Dolin, “On Observing Underwater Objects through a Wavy Water Surface: A New Algorithm for Image Correction and Laboratory Experiment,” Izv., Atmos. Ocean. Phys. 49(3), 339–345 (2013).
[Crossref]

Donate, A.

A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by water waves,” in Proceedings of International Conference on Pattern Recognition (IEEE, 2006), pp. 421–424.

Flacco, N. L.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Fleet, E.

A. V. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “Imaging Through the Air-Water Interface,” in Proceedings of Computational Optical Sensing and Imaging (OSA, 2009), pp. 13–15.

Fraser, D.

Z. Wen, A. Lambert, D. Fraser, and H. Li, “Bispectral analysis and recovery of images distorted by a moving water surface,” Appl. Opt. 49(33), 6376–6384 (2010).
[Crossref] [PubMed]

Z. Wen, A. Lambert, and D. Fraser, “Reconstruction of Imagery Reflected from Water Surface,” inSignal Recovery and Synthesis (OSA, 2007), pp. 18–20.

Z. Wen and D. Fraser, A. lambert and H. Li, “Reconstruction of Underwater Image by Bispectrum,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. 545–548.

Gabarda, S.

Gladysz, S.

Hajnal, J. V.

J. V. Hajnal, N. Saeed, E. J. Soar, A. Oatridge, I. R. Young, and G. M. Bydder, “A Registration and Interpolation Procedure for Subvoxel Matching of Serially Acquired MR Images,” J. Comput. Assist. Tomogr. 19(2), 289–296 (1995).
[Crossref] [PubMed]

Halder, K. K.

Hawkes, D. J.

D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Med. Imaging 18(8), 712–721 (1999).
[Crossref] [PubMed]

Hayes, C.

D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Med. Imaging 18(8), 712–721 (1999).
[Crossref] [PubMed]

He, D.

Hill, D. L. G.

D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Med. Imaging 18(8), 712–721 (1999).
[Crossref] [PubMed]

Hou, W.

Huang, L.

Hubbard, B. E.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Jones, N. M.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Kanaev, A. V.

A. V. Kanaev, W. Hou, S. R. Restaino, S. Matt, and S. Gładysz, “Restoration of images degraded by underwater turbulence using structure tensor oriented image quality (STOIQ) metric,” Opt. Express 23(13), 17077–17090 (2015).
[Crossref] [PubMed]

A. V. Kanaev, W. Hou, S. Woods, and L. N. Smith, “Restoration of turbulence degraded underwater images,” Opt. Eng. 51(5), 057007 (2012).
[Crossref]

A. V. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “Imaging Through the Air-Water Interface,” in Proceedings of Computational Optical Sensing and Imaging (OSA, 2009), pp. 13–15.

Lambert, A.

Z. Wen, A. Lambert, D. Fraser, and H. Li, “Bispectral analysis and recovery of images distorted by a moving water surface,” Appl. Opt. 49(33), 6376–6384 (2010).
[Crossref] [PubMed]

Z. Wen, A. Lambert, and D. Fraser, “Reconstruction of Imagery Reflected from Water Surface,” inSignal Recovery and Synthesis (OSA, 2007), pp. 18–20.

Leach, M. O.

D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Med. Imaging 18(8), 712–721 (1999).
[Crossref] [PubMed]

Lee, S.

S. Lee, G. Wolberg, and S. Y. Shin, “Scattered data interpolation with multilevel B-splines,” IEEE Trans. Vis. Comput. Graph. 3(3), 228–244 (1997).
[Crossref]

Li, H.

Li, L.

L. Li, Q. Wang, and Z. Xiao, “Underwater Image Restoration Algorithm from Distorted Video,” J. Syst. Simul. 24(1), 188–196 (2012).

Li, Y.

Matt, S.

Milder, D. M.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Munk, W.

C. Cox and W. Munk, “Slopes of the sea surface deduced from photographs of sun glitter,” Bull. Scripps Inst. Oceanogr. 6, 401–479 (1956).

Murase, H.

H. Murase, “Surface shape reconstruction of a nonrigid transparent object using refraction and motion,” IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 1045–1052 (1992).
[Crossref]

Murshed, M.

Narasimhan, S. G.

Y. Tian and S. G. Narasimhan, “Theory and Practice of Hierarchical Data-driven Descent for Optimal Deformation Estimation,” Int. J. Comput. Vis. 115(1), 44–67 (2015).
[Crossref]

Y. Tian and S. G. Narasimhan, “Globally Optimal Estimation of Nonrigid Image Distortion,” Int. J. Comput. Vis. 98(3), 279–302 (2012).
[Crossref]

Y. Tian and S. G. Narasimhan, “Seeing through water: image restoration using model-based tracking,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 2303–2310.
[Crossref]

Oatridge, A.

J. V. Hajnal, N. Saeed, E. J. Soar, A. Oatridge, I. R. Young, and G. M. Bydder, “A Registration and Interpolation Procedure for Subvoxel Matching of Serially Acquired MR Images,” J. Comput. Assist. Tomogr. 19(2), 289–296 (1995).
[Crossref] [PubMed]

Oreifej, O.

O. Oreifej, G. Shu, T. Pace, and M. Shah, “A two-stage reconstruction approach for seeing through water,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 1153–1160.
[Crossref]

Pace, T.

O. Oreifej, G. Shu, T. Pace, and M. Shah, “A two-stage reconstruction approach for seeing through water,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 1153–1160.
[Crossref]

Panici, K. R.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Paul, M.

Paul, N.

N. Paul, A. de Chillaz, and J.-L. Collette, “On-line restoration for turbulence degraded video in nuclear power plant reactors,” Signal Image Video Process. 9(3), 601–610 (2015).
[Crossref]

Platt, B. D.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Potter, R. E.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Rajagopalan, A. N.

K. Seemakurthy and A. N. Rajagopalan, “Deskewing of Underwater Images,” IEEE Trans. Image Process. 24(3), 1046–1059 (2015).
[Crossref] [PubMed]

Restaino, S. R.

Ribeiro, E.

A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by water waves,” in Proceedings of International Conference on Pattern Recognition (IEEE, 2006), pp. 421–424.

Rueckert, D.

D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Med. Imaging 18(8), 712–721 (1999).
[Crossref] [PubMed]

Saeed, N.

J. V. Hajnal, N. Saeed, E. J. Soar, A. Oatridge, I. R. Young, and G. M. Bydder, “A Registration and Interpolation Procedure for Subvoxel Matching of Serially Acquired MR Images,” J. Comput. Assist. Tomogr. 19(2), 289–296 (1995).
[Crossref] [PubMed]

Schechner, Y. Y.

T. Treibitz, Y. Y. Schechner, and H. Singh, “Flat refractive geometry,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008).

Scribner, D.

A. V. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “Imaging Through the Air-Water Interface,” in Proceedings of Computational Optical Sensing and Imaging (OSA, 2009), pp. 13–15.

Seemakurthy, K.

K. Seemakurthy and A. N. Rajagopalan, “Deskewing of Underwater Images,” IEEE Trans. Image Process. 24(3), 1046–1059 (2015).
[Crossref] [PubMed]

Shah, M.

O. Oreifej, G. Shu, T. Pace, and M. Shah, “A two-stage reconstruction approach for seeing through water,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 1153–1160.
[Crossref]

Sheikh, H. R.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

Shin, S. Y.

S. Lee, G. Wolberg, and S. Y. Shin, “Scattered data interpolation with multilevel B-splines,” IEEE Trans. Vis. Comput. Graph. 3(3), 228–244 (1997).
[Crossref]

Shu, G.

O. Oreifej, G. Shu, T. Pace, and M. Shah, “A two-stage reconstruction approach for seeing through water,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 1153–1160.
[Crossref]

Simoncelli, E. P.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

Singh, H.

T. Treibitz, Y. Y. Schechner, and H. Singh, “Flat refractive geometry,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008).

Smith, L. N.

A. V. Kanaev, W. Hou, S. Woods, and L. N. Smith, “Restoration of turbulence degraded underwater images,” Opt. Eng. 51(5), 057007 (2012).
[Crossref]

Soar, E. J.

J. V. Hajnal, N. Saeed, E. J. Soar, A. Oatridge, I. R. Young, and G. M. Bydder, “A Registration and Interpolation Procedure for Subvoxel Matching of Serially Acquired MR Images,” J. Comput. Assist. Tomogr. 19(2), 289–296 (1995).
[Crossref] [PubMed]

Sonoda, L. I.

D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Med. Imaging 18(8), 712–721 (1999).
[Crossref] [PubMed]

Tahtali, M.

Tian, Y.

Y. Tian and S. G. Narasimhan, “Theory and Practice of Hierarchical Data-driven Descent for Optimal Deformation Estimation,” Int. J. Comput. Vis. 115(1), 44–67 (2015).
[Crossref]

Y. Tian and S. G. Narasimhan, “Globally Optimal Estimation of Nonrigid Image Distortion,” Int. J. Comput. Vis. 98(3), 279–302 (2012).
[Crossref]

Y. Tian and S. G. Narasimhan, “Seeing through water: image restoration using model-based tracking,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 2303–2310.
[Crossref]

Tong, K. W.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Treibitz, T.

T. Treibitz, Y. Y. Schechner, and H. Singh, “Flat refractive geometry,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008).

Turlaev, D. G.

D. G. Turlaev and L. S. Dolin, “On Observing Underwater Objects through a Wavy Water Surface: A New Algorithm for Image Correction and Laboratory Experiment,” Izv., Atmos. Ocean. Phys. 49(3), 339–345 (2013).
[Crossref]

Twisselmann, D. J.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Wang, Q.

L. Li, Q. Wang, and Z. Xiao, “Underwater Image Restoration Algorithm from Distorted Video,” J. Syst. Simul. 24(1), 188–196 (2012).

Wang, Z.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

Wen, Z.

Z. Wen, A. Lambert, D. Fraser, and H. Li, “Bispectral analysis and recovery of images distorted by a moving water surface,” Appl. Opt. 49(33), 6376–6384 (2010).
[Crossref] [PubMed]

Z. Wen and D. Fraser, A. lambert and H. Li, “Reconstruction of Underwater Image by Bispectrum,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. 545–548.

Z. Wen, A. Lambert, and D. Fraser, “Reconstruction of Imagery Reflected from Water Surface,” inSignal Recovery and Synthesis (OSA, 2007), pp. 18–20.

Wolberg, G.

S. Lee, G. Wolberg, and S. Y. Shin, “Scattered data interpolation with multilevel B-splines,” IEEE Trans. Vis. Comput. Graph. 3(3), 228–244 (1997).
[Crossref]

Woods, S.

A. V. Kanaev, W. Hou, S. Woods, and L. N. Smith, “Restoration of turbulence degraded underwater images,” Opt. Eng. 51(5), 057007 (2012).
[Crossref]

Xiao, Z.

L. Li, Q. Wang, and Z. Xiao, “Underwater Image Restoration Algorithm from Distorted Video,” J. Syst. Simul. 24(1), 188–196 (2012).

Young, I. R.

J. V. Hajnal, N. Saeed, E. J. Soar, A. Oatridge, I. R. Young, and G. M. Bydder, “A Registration and Interpolation Procedure for Subvoxel Matching of Serially Acquired MR Images,” J. Comput. Assist. Tomogr. 19(2), 289–296 (1995).
[Crossref] [PubMed]

Zhang, R.

Appl. Opt. (3)

Bull. Scripps Inst. Oceanogr. (1)

C. Cox and W. Munk, “Slopes of the sea surface deduced from photographs of sun glitter,” Bull. Scripps Inst. Oceanogr. 6, 401–479 (1956).

IEEE Trans. Image Process. (2)

K. Seemakurthy and A. N. Rajagopalan, “Deskewing of Underwater Images,” IEEE Trans. Image Process. 24(3), 1046–1059 (2015).
[Crossref] [PubMed]

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

IEEE Trans. Med. Imaging (1)

D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images,” IEEE Trans. Med. Imaging 18(8), 712–721 (1999).
[Crossref] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

H. Murase, “Surface shape reconstruction of a nonrigid transparent object using refraction and motion,” IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 1045–1052 (1992).
[Crossref]

IEEE Trans. Vis. Comput. Graph. (1)

S. Lee, G. Wolberg, and S. Y. Shin, “Scattered data interpolation with multilevel B-splines,” IEEE Trans. Vis. Comput. Graph. 3(3), 228–244 (1997).
[Crossref]

Int. J. Comput. Vis. (2)

Y. Tian and S. G. Narasimhan, “Globally Optimal Estimation of Nonrigid Image Distortion,” Int. J. Comput. Vis. 98(3), 279–302 (2012).
[Crossref]

Y. Tian and S. G. Narasimhan, “Theory and Practice of Hierarchical Data-driven Descent for Optimal Deformation Estimation,” Int. J. Comput. Vis. 115(1), 44–67 (2015).
[Crossref]

Izv., Atmos. Ocean. Phys. (1)

D. G. Turlaev and L. S. Dolin, “On Observing Underwater Objects through a Wavy Water Surface: A New Algorithm for Image Correction and Laboratory Experiment,” Izv., Atmos. Ocean. Phys. 49(3), 339–345 (2013).
[Crossref]

J. Comput. Assist. Tomogr. (1)

J. V. Hajnal, N. Saeed, E. J. Soar, A. Oatridge, I. R. Young, and G. M. Bydder, “A Registration and Interpolation Procedure for Subvoxel Matching of Serially Acquired MR Images,” J. Comput. Assist. Tomogr. 19(2), 289–296 (1995).
[Crossref] [PubMed]

J. Opt. Soc. Am. A (2)

J. Syst. Simul. (1)

L. Li, Q. Wang, and Z. Xiao, “Underwater Image Restoration Algorithm from Distorted Video,” J. Syst. Simul. 24(1), 188–196 (2012).

Opt. Eng. (1)

A. V. Kanaev, W. Hou, S. Woods, and L. N. Smith, “Restoration of turbulence degraded underwater images,” Opt. Eng. 51(5), 057007 (2012).
[Crossref]

Opt. Express (2)

Signal Image Video Process. (1)

N. Paul, A. de Chillaz, and J.-L. Collette, “On-line restoration for turbulence degraded video in nuclear power plant reactors,” Signal Image Video Process. 9(3), 601–610 (2015).
[Crossref]

Waves Random Complex Media (1)

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Other (9)

T. Treibitz, Y. Y. Schechner, and H. Singh, “Flat refractive geometry,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2008).

Z. Wen and D. Fraser, A. lambert and H. Li, “Reconstruction of Underwater Image by Bispectrum,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2007), pp. 545–548.

Z. Wen, A. Lambert, and D. Fraser, “Reconstruction of Imagery Reflected from Water Surface,” inSignal Recovery and Synthesis (OSA, 2007), pp. 18–20.

Y. Tian and S. G. Narasimhan, “Seeing through water: image restoration using model-based tracking,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 2303–2310.
[Crossref]

O. Oreifej, G. Shu, T. Pace, and M. Shah, “A two-stage reconstruction approach for seeing through water,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 1153–1160.
[Crossref]

A. Efros, V. Isler, J. Shi, and M. Visontai, “Seeing through water,” in Proceedings of Neural Information Processing Systems (Neural Information Processing Systems Foundation, 2004), pp. 393–400.

A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by water waves,” in Proceedings of International Conference on Pattern Recognition (IEEE, 2006), pp. 421–424.

K. K. Halder, M. Tahtali, and S. G. Anavatti, “An Artificial Neural Network Approach for Underwater Warp Prediction,” in Proceedings of Hellenic Conference on Artificial Intelligence (ACM, 2014), pp. 384–394.
[Crossref]

A. V. Kanaev, J. Ackerman, E. Fleet, and D. Scribner, “Imaging Through the Air-Water Interface,” in Proceedings of Computational Optical Sensing and Imaging (OSA, 2009), pp. 13–15.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1 Simple block diagram of the proposed image restoration method.
Fig. 2
Fig. 2 Sample frame and mean of each data set.
Fig. 3
Fig. 3 The mean results of the two methods after different iterations of the registration process. The regions with larger distortions are marked with red rectangles.
Fig. 4
Fig. 4 Sample frame and mean of our data set.
Fig. 5
Fig. 5 The mean results of the two methods with our data set. The regions with larger distortions are marked with red rectangles.
Fig. 6
Fig. 6 The mean after applying five iterations of registration. (a) Use the mean without deblurring as the reference (standard mean-to-frame registration). (b) Use the mean deblurred by the blind deconvolution as the reference.
Fig. 7
Fig. 7 Mean results with different image quality metrics.

Tables (3)

Tables Icon

Table 1 The complete restoration strategy is summarized in algorithm 1.

Tables Icon

Table 1 Number of Iterations When the Robust Registration Process Stops

Tables Icon

Table 2 Comparison of Image Quality Metrics among the Proposed Method, the Oreifej method [5] and the Tian method [4]a

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

SSIM(I,M)= ( 2 μ I μ M + C 1 )( 2 σ IM + C 2 ) ( μ I 2 + μ M 2 + C 1 )+( σ I 2 + σ M 2 + C 2 ) ,
f k+1 = f k ( h k g h k f k )Ψ( f k ),
b k = a k + λ k δ k
T(x,y)= m=0 3 l=0 3 B m (v) B l (u) ϕ i+l,j+m ,
E smooth = 1 A 0 X 0 Y [ ( 2 T x 2 ) 2 + ( 2 T y 2 ) 2 +2 ( 2 T xy ) 2 ] dxdy,
E similarity = (x,y)Ω ( I A ( x,y ) I B ( x,y ) ) 2 w×h ,
E(Φ)= E similarity +α E smooth .
l 1 (V)= k (x,y) | M(x,y) I k (x,y) | w×h×n .
MSE= (x,y) ( M(x,y)G(x,y) ) 2 w×h ,
PSNR=10 log 10 ( ( MAX(G) ) 2 MSE ),

Metrics