Abstract

A novel algorithm is proposed to predict “lucky” regions in a sequence of long-range camera images affected by atmospheric turbulence. Our new approach is to employ bicoherence as a measure of quality to determine lucky regions or good quality image patches from a recorded sequence of anisoplanatic images. The better-quality image regions are selected according to the magnitude of the average value of the bicoherence of each region. Each image patch is restored using bispectral phase estimation from lucky regions, before mosaicing to an overall restoration. Bicoherence can also be used to predict lucky images in the isoplanatic case. Experiments show that our algorithm performs well with both simulated and naturally degraded data.

© 2009 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. Roggemann and B. Welch, Imaging Through Turbulence (CRC, 1996).
  2. C. J. Carrano, “Speckle imaging over horizontal paths,” Proc. SPIE 4825, 65-75 (2002).
  3. D. Fried, “Probability of getting a lucky short-exposure image through turbulence,” J. Opt. Soc. Am. A 68, 1651-1658 (1978).
    [CrossRef]
  4. R. Tubbs, “Lucky exposures: diffraction limited astronomical imaging through the atmosphere,” Ph.D. thesis (Cambridge University, 2003).
  5. N. Law, C. Mackay, and J. Baldwin, “Lucky imaging: high angular resolution imaging in the visible from the ground,” Astron. Astrophys. 446, 739-745 (2006).
    [CrossRef]
  6. S. Weddell and R. Webb, “Data preprocessing on sequential data for improved astronomical imaging,” in Proceedings of Image and Vision Computing (Academic, 2005).
  7. M. Vorontsov and G. Carhart, “Anisoplanatic imaging through turbulent media: image recovery by local information fusion from a set of short-exposure images,” J. Opt. Soc. Am. A 18, 1312-1324 (2001).
    [CrossRef]
  8. J. Fackrell and S. McLaughlin, “Quadratic phase coupling detection using higher order statistics,” in Proceedings of the IEE Colloquium on Higher Order Statistics (Academic, 1995), pp. 9/1-9/8.
  9. J. Mendel, “Tutorial on higher-order statistics (spectra) in signal processing and system theory: theoretical results and some applications,” Proc. IEEE 79, 278-305 (1991).
  10. C. Nikias and A. Petropulu, Higher-Order Spectra Analysis (Prentice-Hall, 1993).
  11. A. Lohmann and B. Wirnitzer, “Triple correlations,” in Proc. IEEE 72, 889-901 (1984).
    [CrossRef]
  12. T. D. de Wit, “Spectral and statistical analysis of plasma turbulence: beyond linear techniques,” in Space Plasma Simulation (Springer, 2003).
  13. W. Silva, T. Strganac, and M. Hajj, “Higher-order spectral analysis of a nonlinear pitch and plunge apparatus,” in Proceedings of the 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. (Academic, 2005), pp. 2005-2013.
  14. S. McLaughlin, A. Stogioglou, and J. Fackrell, “Introducing higher order statistics (HOS) for the detection of nonlinearities,” UK Nonlinear News (1995).
  15. M. Hinich and M. Wolinsky, “Normalizing bispectra,” J. Statistical Planning Inference 130, 405-411 (2005).
    [CrossRef]
  16. T. Ng, S. Chang, and Q. Sun, “Blind detection of photomontage using higher order statistics,” in Proceedings of the International Symposium on Circuits and Systems, ISCAS 2004, Vol. 5 (IEEE, 2004), pp. 688-691.
  17. H. Farid and A. Popescu, “Blind removal of image nonlinearities,” in Proceedings of IEEE Conference on International Conference of Computer Vision (IEEE, 2001), pp. 76-81.
  18. Z. Wen, D. Fraser, and A. Lambert, “Bicoherence Used to Predict Lucky Regions in Turbulence Affected Surveillance,” in IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS'06) (2006), pp. 108-114.
    [CrossRef]
  19. C. Matson, “Weighted-least-squares phase reconstruction from the bispectrum,” J. Opt. Soc. Am. A 8, 1905-1913 (1991).
    [CrossRef]
  20. Bovik, C. Alan, “New directions in image and video quality assessment plenary talk,” Proceedings of IEEE 9th Workshop on Multimedia Signal Processing (MMSP) (2007).
  21. C. M. Harding, R. A. Johnston, and R. G. Lane, “Fast simulation of a Kolmogorov phase screen,” Appl. Opt. 38, 2161-2170 (1999).
    [CrossRef]
  22. A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65-75 (2004).
    [CrossRef]
  23. C. J. Carrano, “Anisoplanatic performance of horizontal-path speckle imaging,” Proc. SPIE 5162, 14-26 (2003).
    [CrossRef]

2006 (1)

N. Law, C. Mackay, and J. Baldwin, “Lucky imaging: high angular resolution imaging in the visible from the ground,” Astron. Astrophys. 446, 739-745 (2006).
[CrossRef]

2005 (1)

M. Hinich and M. Wolinsky, “Normalizing bispectra,” J. Statistical Planning Inference 130, 405-411 (2005).
[CrossRef]

2004 (1)

A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65-75 (2004).
[CrossRef]

2003 (1)

C. J. Carrano, “Anisoplanatic performance of horizontal-path speckle imaging,” Proc. SPIE 5162, 14-26 (2003).
[CrossRef]

2002 (1)

C. J. Carrano, “Speckle imaging over horizontal paths,” Proc. SPIE 4825, 65-75 (2002).

2001 (1)

1999 (1)

1991 (1)

1984 (1)

A. Lohmann and B. Wirnitzer, “Triple correlations,” in Proc. IEEE 72, 889-901 (1984).
[CrossRef]

1978 (1)

D. Fried, “Probability of getting a lucky short-exposure image through turbulence,” J. Opt. Soc. Am. A 68, 1651-1658 (1978).
[CrossRef]

Alan, C.

Bovik, C. Alan, “New directions in image and video quality assessment plenary talk,” Proceedings of IEEE 9th Workshop on Multimedia Signal Processing (MMSP) (2007).

Baldwin, J.

N. Law, C. Mackay, and J. Baldwin, “Lucky imaging: high angular resolution imaging in the visible from the ground,” Astron. Astrophys. 446, 739-745 (2006).
[CrossRef]

Bovik,

Bovik, C. Alan, “New directions in image and video quality assessment plenary talk,” Proceedings of IEEE 9th Workshop on Multimedia Signal Processing (MMSP) (2007).

Carhart, G.

Carrano, C. J.

C. J. Carrano, “Anisoplanatic performance of horizontal-path speckle imaging,” Proc. SPIE 5162, 14-26 (2003).
[CrossRef]

C. J. Carrano, “Speckle imaging over horizontal paths,” Proc. SPIE 4825, 65-75 (2002).

Chang, S.

T. Ng, S. Chang, and Q. Sun, “Blind detection of photomontage using higher order statistics,” in Proceedings of the International Symposium on Circuits and Systems, ISCAS 2004, Vol. 5 (IEEE, 2004), pp. 688-691.

de Wit, T. D.

T. D. de Wit, “Spectral and statistical analysis of plasma turbulence: beyond linear techniques,” in Space Plasma Simulation (Springer, 2003).

Fackrell, J.

S. McLaughlin, A. Stogioglou, and J. Fackrell, “Introducing higher order statistics (HOS) for the detection of nonlinearities,” UK Nonlinear News (1995).

J. Fackrell and S. McLaughlin, “Quadratic phase coupling detection using higher order statistics,” in Proceedings of the IEE Colloquium on Higher Order Statistics (Academic, 1995), pp. 9/1-9/8.

Farid, H.

H. Farid and A. Popescu, “Blind removal of image nonlinearities,” in Proceedings of IEEE Conference on International Conference of Computer Vision (IEEE, 2001), pp. 76-81.

Fraser, D.

A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65-75 (2004).
[CrossRef]

Z. Wen, D. Fraser, and A. Lambert, “Bicoherence Used to Predict Lucky Regions in Turbulence Affected Surveillance,” in IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS'06) (2006), pp. 108-114.
[CrossRef]

Fried, D.

D. Fried, “Probability of getting a lucky short-exposure image through turbulence,” J. Opt. Soc. Am. A 68, 1651-1658 (1978).
[CrossRef]

Hajj, M.

W. Silva, T. Strganac, and M. Hajj, “Higher-order spectral analysis of a nonlinear pitch and plunge apparatus,” in Proceedings of the 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. (Academic, 2005), pp. 2005-2013.

Harding, C. M.

Hinich, M.

M. Hinich and M. Wolinsky, “Normalizing bispectra,” J. Statistical Planning Inference 130, 405-411 (2005).
[CrossRef]

Johnston, R. A.

Lambert, A.

Z. Wen, D. Fraser, and A. Lambert, “Bicoherence Used to Predict Lucky Regions in Turbulence Affected Surveillance,” in IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS'06) (2006), pp. 108-114.
[CrossRef]

Lambert, A. J.

A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65-75 (2004).
[CrossRef]

Lane, R. G.

Law, N.

N. Law, C. Mackay, and J. Baldwin, “Lucky imaging: high angular resolution imaging in the visible from the ground,” Astron. Astrophys. 446, 739-745 (2006).
[CrossRef]

Lohmann, A.

A. Lohmann and B. Wirnitzer, “Triple correlations,” in Proc. IEEE 72, 889-901 (1984).
[CrossRef]

Mackay, C.

N. Law, C. Mackay, and J. Baldwin, “Lucky imaging: high angular resolution imaging in the visible from the ground,” Astron. Astrophys. 446, 739-745 (2006).
[CrossRef]

Matson, C.

McLaughlin, S.

J. Fackrell and S. McLaughlin, “Quadratic phase coupling detection using higher order statistics,” in Proceedings of the IEE Colloquium on Higher Order Statistics (Academic, 1995), pp. 9/1-9/8.

S. McLaughlin, A. Stogioglou, and J. Fackrell, “Introducing higher order statistics (HOS) for the detection of nonlinearities,” UK Nonlinear News (1995).

Mendel, J.

J. Mendel, “Tutorial on higher-order statistics (spectra) in signal processing and system theory: theoretical results and some applications,” Proc. IEEE 79, 278-305 (1991).

Ng, T.

T. Ng, S. Chang, and Q. Sun, “Blind detection of photomontage using higher order statistics,” in Proceedings of the International Symposium on Circuits and Systems, ISCAS 2004, Vol. 5 (IEEE, 2004), pp. 688-691.

Nikias, C.

C. Nikias and A. Petropulu, Higher-Order Spectra Analysis (Prentice-Hall, 1993).

Petropulu, A.

C. Nikias and A. Petropulu, Higher-Order Spectra Analysis (Prentice-Hall, 1993).

Popescu, A.

H. Farid and A. Popescu, “Blind removal of image nonlinearities,” in Proceedings of IEEE Conference on International Conference of Computer Vision (IEEE, 2001), pp. 76-81.

Roggemann, M.

M. Roggemann and B. Welch, Imaging Through Turbulence (CRC, 1996).

Silva, W.

W. Silva, T. Strganac, and M. Hajj, “Higher-order spectral analysis of a nonlinear pitch and plunge apparatus,” in Proceedings of the 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. (Academic, 2005), pp. 2005-2013.

Stogioglou, A.

S. McLaughlin, A. Stogioglou, and J. Fackrell, “Introducing higher order statistics (HOS) for the detection of nonlinearities,” UK Nonlinear News (1995).

Strganac, T.

W. Silva, T. Strganac, and M. Hajj, “Higher-order spectral analysis of a nonlinear pitch and plunge apparatus,” in Proceedings of the 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. (Academic, 2005), pp. 2005-2013.

Sun, Q.

T. Ng, S. Chang, and Q. Sun, “Blind detection of photomontage using higher order statistics,” in Proceedings of the International Symposium on Circuits and Systems, ISCAS 2004, Vol. 5 (IEEE, 2004), pp. 688-691.

Tubbs, R.

R. Tubbs, “Lucky exposures: diffraction limited astronomical imaging through the atmosphere,” Ph.D. thesis (Cambridge University, 2003).

Vorontsov, M.

Webb, R.

S. Weddell and R. Webb, “Data preprocessing on sequential data for improved astronomical imaging,” in Proceedings of Image and Vision Computing (Academic, 2005).

Weddell, S.

S. Weddell and R. Webb, “Data preprocessing on sequential data for improved astronomical imaging,” in Proceedings of Image and Vision Computing (Academic, 2005).

Welch, B.

M. Roggemann and B. Welch, Imaging Through Turbulence (CRC, 1996).

Wen, Z.

Z. Wen, D. Fraser, and A. Lambert, “Bicoherence Used to Predict Lucky Regions in Turbulence Affected Surveillance,” in IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS'06) (2006), pp. 108-114.
[CrossRef]

Wirnitzer, B.

A. Lohmann and B. Wirnitzer, “Triple correlations,” in Proc. IEEE 72, 889-901 (1984).
[CrossRef]

Wolinsky, M.

M. Hinich and M. Wolinsky, “Normalizing bispectra,” J. Statistical Planning Inference 130, 405-411 (2005).
[CrossRef]

Appl. Opt. (1)

Astron. Astrophys. (1)

N. Law, C. Mackay, and J. Baldwin, “Lucky imaging: high angular resolution imaging in the visible from the ground,” Astron. Astrophys. 446, 739-745 (2006).
[CrossRef]

J. Opt. Soc. Am. A (3)

J. Statistical Planning Inference (1)

M. Hinich and M. Wolinsky, “Normalizing bispectra,” J. Statistical Planning Inference 130, 405-411 (2005).
[CrossRef]

Proc. IEEE (1)

A. Lohmann and B. Wirnitzer, “Triple correlations,” in Proc. IEEE 72, 889-901 (1984).
[CrossRef]

Proc. SPIE (3)

A. J. Lambert and D. Fraser, “Superresolution in imagery arising from observation through anisoplanatic distortion,” Proc. SPIE 5562, 65-75 (2004).
[CrossRef]

C. J. Carrano, “Anisoplanatic performance of horizontal-path speckle imaging,” Proc. SPIE 5162, 14-26 (2003).
[CrossRef]

C. J. Carrano, “Speckle imaging over horizontal paths,” Proc. SPIE 4825, 65-75 (2002).

Other (13)

S. Weddell and R. Webb, “Data preprocessing on sequential data for improved astronomical imaging,” in Proceedings of Image and Vision Computing (Academic, 2005).

Bovik, C. Alan, “New directions in image and video quality assessment plenary talk,” Proceedings of IEEE 9th Workshop on Multimedia Signal Processing (MMSP) (2007).

R. Tubbs, “Lucky exposures: diffraction limited astronomical imaging through the atmosphere,” Ph.D. thesis (Cambridge University, 2003).

J. Fackrell and S. McLaughlin, “Quadratic phase coupling detection using higher order statistics,” in Proceedings of the IEE Colloquium on Higher Order Statistics (Academic, 1995), pp. 9/1-9/8.

J. Mendel, “Tutorial on higher-order statistics (spectra) in signal processing and system theory: theoretical results and some applications,” Proc. IEEE 79, 278-305 (1991).

C. Nikias and A. Petropulu, Higher-Order Spectra Analysis (Prentice-Hall, 1993).

T. D. de Wit, “Spectral and statistical analysis of plasma turbulence: beyond linear techniques,” in Space Plasma Simulation (Springer, 2003).

W. Silva, T. Strganac, and M. Hajj, “Higher-order spectral analysis of a nonlinear pitch and plunge apparatus,” in Proceedings of the 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. (Academic, 2005), pp. 2005-2013.

S. McLaughlin, A. Stogioglou, and J. Fackrell, “Introducing higher order statistics (HOS) for the detection of nonlinearities,” UK Nonlinear News (1995).

T. Ng, S. Chang, and Q. Sun, “Blind detection of photomontage using higher order statistics,” in Proceedings of the International Symposium on Circuits and Systems, ISCAS 2004, Vol. 5 (IEEE, 2004), pp. 688-691.

H. Farid and A. Popescu, “Blind removal of image nonlinearities,” in Proceedings of IEEE Conference on International Conference of Computer Vision (IEEE, 2001), pp. 76-81.

Z. Wen, D. Fraser, and A. Lambert, “Bicoherence Used to Predict Lucky Regions in Turbulence Affected Surveillance,” in IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS'06) (2006), pp. 108-114.
[CrossRef]

M. Roggemann and B. Welch, Imaging Through Turbulence (CRC, 1996).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1

Bicoherence computed using 1-D lines in the image domain has similar characteristics to that calculated using the 2-D image. The values of the bicoherence of 50 distorted frames are plotted. The solid line presents the values using the 2-D image, while the thicker dotted line is from the 1-D lines in the image domain. Both curves have a similar shape, which gives similar estimation of the quality of one image, but importantly, the vertical ranges of both are co-confined.

Fig. 2
Fig. 2

Comparison of the value of the bicoherence of a true image and of a set of simulated atmospheric turbulence-affected images. A low value implies less effect by the turbulence and hence a low level of phase coupling. The original image of course will have the lowest bicoherence. The threshold that selects lucky frames or regions over more degraded images may be chosen based on the data sequence.

Fig. 3
Fig. 3

Isoplanatic case showing a comparison of different methods of restoration of a set of images having simulated isoplanatic atmospheric distortion: (a) single degraded image, (b) result of the mean of regions having minimum MSE, (c) result by averaging the sequence, (d) result by averaging and mosaicing the lucky regions, (e) result restored by our algorithm, (f) the true image. The result (e) by our algorithm has the best signal-to-noise ratio and exhibits the sharpest and closest match to the true image visually.

Fig. 4
Fig. 4

Anisoplanatic case showing a comparison of different methods of restoration of a set of images having simulated anisoplanatic atmospheric distortion: (a) single degraded image (note scintillation), (b) result of the mean of regions having minimum MSE, (c) result obtained by averaging the sequence, (d) result obtained by averaging and mosaicing the lucky regions, (e) result restored by our algorithm, (f) true image. The result (e) by our algorithm has the best signal-to-noise ratio and exhibits the sharpest and closest match to the true image visually.

Fig. 5
Fig. 5

Results from a time-sequence of real-world data: (a) typical example 128 × 128 pixel image of the sequence, (b) result of the mean of regions having minimum MSE, (c) result obatined by averaging the sequence, (d) result by averaging and mosaicing the lucky regions, (e) result restored by our algorithm. Observe in (e) not only the enhancement around the building structure but also the extraction of texture in the tree foliage.

Fig. 6
Fig. 6

Results with different lucky region ratios. (a) 5%, (b) 20%, (c) 40%, and (d) 60%. For the purpose of detail comparison, only a part of the image is shown. (a) and (d) are more blurred and less geometrically accurate (the edges), which is the result that less information (in 5%) and more ambiguities (in 60%) are introduced using smaller or larger ratios. (c) and (d) are very alike, which indicates that reasonable optimization may happen between 20% and 40%.

Tables (1)

Tables Icon

Table 1 Samples of Input and Their Bicoherence Values

Equations (15)

Equations on this page are rendered with MathJax. Learn more.

v ( x , t ) x = f ( v ( x , t ) ) ,
v ( x , t ) x = g ( τ 1 ) v ( x , t τ 1 ) d τ 1 + g ( τ 1 , τ 2 ) v ( x , t τ 1 ) v ( x , t τ 2 ) d τ 1 τ 2 + g ( τ 1 , τ 2 , τ 3 ) v ( x , t τ 1 ) v ( x , t τ 2 ) × v ( x , t τ 3 ) d τ 1 τ 2 τ 3 + .
v p x = Γ p v p + m , n Γ m n v m v n δ m + n , p + m , n , k Γ m n k v m v n v k δ m + n + k , p + .
u p = u m + u n .
I ( u ) = x = 0 N x 1 i ( x ) e j 2 π u x N x ,
i B ( x 1 , x 2 ) = x = 0 N x 1 i ( x ) i ( x + x 1 ) i ( x + x 2 ) ,
I B ( u 1 , u 2 ) = I ( u 1 ) I ( u 2 ) I * ( u 1 + u 2 ) = I ( u 1 ) I ( u 2 ) I ( u 1 u 2 ) ,
b ( u 1 , u 2 ) = | 1 M [ s = 0 M 1 I s ( u 1 ) I s ( u 2 ) I s * ( u 1 + u 2 ) ] | 1 M [ s = 0 M 1 | I s ( u 1 ) I s ( u 2 ) | 2 ] · 1 M [ s = 0 M 1 | I s * ( u 1 + u 2 ) | 2 ] ,
b h = 1 N h k = 1 N h [ 1 N u 1 N u 2 u 1 = 0 N u 1 1 u 2 = 0 N u 2 1 b k hor _ line ( u 1 , u 2 ) ] ,
b v = 1 N v k = 1 N υ [ 1 N u 1 N u 2 u 1 = 0 N u 1 1 u 2 = 0 N u 2 1 b k ver _ line ( u 1 , u 2 ) ] ,
b c = b h 2 + b v 2 ,
ϕ B ( u 1 , u 2 ) = ϕ I ( u 1 ) + ϕ I ( u 2 ) ϕ I ( u 1 + u 2 ) ,
I k ( u , v ) = I ( u , v ) e j ϕ k ( u , v ) ,
θ 0 = 0.314 r 0 L scr ,
PSNR = 20 log ( A image A noise ) ,

Metrics