Abstract

Optical image tracing is one of key technologies to realize and maintain satellite-to-ground laser communication. Since machine learning has been proved to be a powerful tool for modeling nonlinear system, a model containing a preprocessing module, a CNN module (Convolutional Neural Network Module) as well as a LSTM module (Long-Short Term Neural Network Memory Module) was developed to process digital images in time series and then predict centroid positions under the influence of atmospheric turbulence. Different from most previous models composed of neural networks, some important physical situations are considered for light fields distributed on CMOS. By building and training this model, centroid positions can be predicted in real time for practical applications in laser satellite communication.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Object-independent image-based wavefront sensing approach using phase diversity images and deep learning

Qi Xin, Guohao Ju, Chunyue Zhang, and Shuyan Xu
Opt. Express 27(18) 26102-26119 (2019)

OSNR and nonlinear noise power estimation for optical fiber communication systems using LSTM based deep learning technique

Ziyi Wang, Aiying Yang, Peng Guo, and Pinjing He
Opt. Express 26(16) 21346-21357 (2018)

Deep learning based atmospheric turbulence compensation for orbital angular momentum beam distortion and communication

Junmin Liu, Peipei Wang, Xiaoke Zhang, Yanliang He, Xinxing Zhou, Huapeng Ye, Ying Li, Shixiang Xu, Shuqing Chen, and Dianyuan Fan
Opt. Express 27(12) 16671-16688 (2019)

References

  • View by:
  • |
  • |
  • |

  1. D. Cornwell, “Space-Based Laser Communications Break Threshold,” Opt. Photonics News 27(5), 24–31 (2016).
    [Crossref]
  2. M. R. Clark, “Application of Machine Learning Principles to Modeling of Nonlinear Dynamic Systems,” J. Arkansas Acad. Sci. 48, 36–40 (1994).
  3. K. Worden and P. L. Green, “A machine learning approach to nonlinear modal analysis,” Mech. Syst. Signal. Process. 84, 34–53 (2017).
    [Crossref]
  4. G. Ju, X. Qi, H. Ma, and C. Yan, “Feature-based phase retrieval wavefront sensing approach using machine learning,” Opt. Express 26(24), 31767–31783 (2018).
    [Crossref]
  5. D. Margaritis, “Learning Bayesian Network Model Structure from Data,” PhD dissertation, Carnegie Mellon University, (2013)
  6. S. Arnon and N. S. Kopeika, “Adaptive suboptimum detection of an optical pulse-position-modulation signal with a detection matrix and centroid tracking,” J. Opt. Soc. Am. A 15(2), 443–448 (1998).
    [Crossref]
  7. P. Zarchan and H. Musoff, Fundamentals of Kalman Filtering (Progress in Aeronautics and Astronautics) (AIAA, 2015), pp. 210–212
  8. G. Farnebäck, “Two-Frame Motion Estimation Based on Polynomial Expansion,” in Proceedings of the 13th Scandinavian Conference on Image Analysis (SCIA, 2003), pp. 363–370.
  9. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation 9(8), 1735–1780 (1997).
    [Crossref]
  10. R. C. Gonzalez and R. E. Woods, Digital Image Processing (Pearson, 2017), pp. 57–120.
  11. J. W. Goodman, Introduction to Fourier Optics (W.H. Freeman, 2017), pp. 441–452.
  12. R. Y. Tsai, “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses,” IEEE J. Robot. Automat. 3(4), 323–344 (1987).
    [Crossref]
  13. O. Keller, Light - The Physics of the Photon (CRC Press, 2014), pp. 378–402.
  14. K. Cahill, Physical Mathematics (Cambridge University Press, 2013), pp. 245–256.
  15. J. Corso, “Motion and Optical Flow,” https://web.eecs.umich.edu/∼jjcorso/t/598F14/files/lecture_1015_motion.pdf , University of Chicago (2014).
  16. Q. Wang, Y. Siyuan, L. Tan, and J. Ma, “Approach for Recognizing and Tracking Beacon in Inter-Satellite Optical Communication Based on Optical Flow Method,” Opt. Express 26(21), 28080–28090 (2018).
    [Crossref]
  17. D. B. Bungbung and D. Valero, “Application of the Optical Flow Method to Velocity Determination in Hydraulic Structure Models,” in 6th International Symposium on Hydraulic Structures and Water System Management, Portland, (2016).
  18. E. R. Davies, Computer Vision: Principles, Algorithms, Applications, Learning (Academic Press, 2017), pp. 347–357.
  19. L. C. Andrews and R. L. Phillips, Laser Beam Propagation through Random Media (SPIE Press, 1998).
  20. https://docs.python.org/2/library/multiprocessing.html
  21. https://docs.python.org/3.8/library/multiprocessing.shared_memory.html
  22. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (The MIT Press, 2016), pp. 542–550.

2018 (2)

2017 (1)

K. Worden and P. L. Green, “A machine learning approach to nonlinear modal analysis,” Mech. Syst. Signal. Process. 84, 34–53 (2017).
[Crossref]

2016 (1)

D. Cornwell, “Space-Based Laser Communications Break Threshold,” Opt. Photonics News 27(5), 24–31 (2016).
[Crossref]

1998 (1)

1997 (1)

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation 9(8), 1735–1780 (1997).
[Crossref]

1994 (1)

M. R. Clark, “Application of Machine Learning Principles to Modeling of Nonlinear Dynamic Systems,” J. Arkansas Acad. Sci. 48, 36–40 (1994).

1987 (1)

R. Y. Tsai, “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses,” IEEE J. Robot. Automat. 3(4), 323–344 (1987).
[Crossref]

Andrews, L. C.

L. C. Andrews and R. L. Phillips, Laser Beam Propagation through Random Media (SPIE Press, 1998).

Arnon, S.

Bengio, Y.

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (The MIT Press, 2016), pp. 542–550.

Bungbung, D. B.

D. B. Bungbung and D. Valero, “Application of the Optical Flow Method to Velocity Determination in Hydraulic Structure Models,” in 6th International Symposium on Hydraulic Structures and Water System Management, Portland, (2016).

Cahill, K.

K. Cahill, Physical Mathematics (Cambridge University Press, 2013), pp. 245–256.

Clark, M. R.

M. R. Clark, “Application of Machine Learning Principles to Modeling of Nonlinear Dynamic Systems,” J. Arkansas Acad. Sci. 48, 36–40 (1994).

Cornwell, D.

D. Cornwell, “Space-Based Laser Communications Break Threshold,” Opt. Photonics News 27(5), 24–31 (2016).
[Crossref]

Corso, J.

J. Corso, “Motion and Optical Flow,” https://web.eecs.umich.edu/∼jjcorso/t/598F14/files/lecture_1015_motion.pdf , University of Chicago (2014).

Courville, A.

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (The MIT Press, 2016), pp. 542–550.

Davies, E. R.

E. R. Davies, Computer Vision: Principles, Algorithms, Applications, Learning (Academic Press, 2017), pp. 347–357.

Farnebäck, G.

G. Farnebäck, “Two-Frame Motion Estimation Based on Polynomial Expansion,” in Proceedings of the 13th Scandinavian Conference on Image Analysis (SCIA, 2003), pp. 363–370.

Gonzalez, R. C.

R. C. Gonzalez and R. E. Woods, Digital Image Processing (Pearson, 2017), pp. 57–120.

Goodfellow, I.

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (The MIT Press, 2016), pp. 542–550.

Goodman, J. W.

J. W. Goodman, Introduction to Fourier Optics (W.H. Freeman, 2017), pp. 441–452.

Green, P. L.

K. Worden and P. L. Green, “A machine learning approach to nonlinear modal analysis,” Mech. Syst. Signal. Process. 84, 34–53 (2017).
[Crossref]

Hochreiter, S.

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation 9(8), 1735–1780 (1997).
[Crossref]

Ju, G.

Keller, O.

O. Keller, Light - The Physics of the Photon (CRC Press, 2014), pp. 378–402.

Kopeika, N. S.

Ma, H.

Ma, J.

Margaritis, D.

D. Margaritis, “Learning Bayesian Network Model Structure from Data,” PhD dissertation, Carnegie Mellon University, (2013)

Musoff, H.

P. Zarchan and H. Musoff, Fundamentals of Kalman Filtering (Progress in Aeronautics and Astronautics) (AIAA, 2015), pp. 210–212

Phillips, R. L.

L. C. Andrews and R. L. Phillips, Laser Beam Propagation through Random Media (SPIE Press, 1998).

Qi, X.

Schmidhuber, J.

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation 9(8), 1735–1780 (1997).
[Crossref]

Siyuan, Y.

Tan, L.

Tsai, R. Y.

R. Y. Tsai, “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses,” IEEE J. Robot. Automat. 3(4), 323–344 (1987).
[Crossref]

Valero, D.

D. B. Bungbung and D. Valero, “Application of the Optical Flow Method to Velocity Determination in Hydraulic Structure Models,” in 6th International Symposium on Hydraulic Structures and Water System Management, Portland, (2016).

Wang, Q.

Woods, R. E.

R. C. Gonzalez and R. E. Woods, Digital Image Processing (Pearson, 2017), pp. 57–120.

Worden, K.

K. Worden and P. L. Green, “A machine learning approach to nonlinear modal analysis,” Mech. Syst. Signal. Process. 84, 34–53 (2017).
[Crossref]

Yan, C.

Zarchan, P.

P. Zarchan and H. Musoff, Fundamentals of Kalman Filtering (Progress in Aeronautics and Astronautics) (AIAA, 2015), pp. 210–212

IEEE J. Robot. Automat. (1)

R. Y. Tsai, “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses,” IEEE J. Robot. Automat. 3(4), 323–344 (1987).
[Crossref]

J. Arkansas Acad. Sci. (1)

M. R. Clark, “Application of Machine Learning Principles to Modeling of Nonlinear Dynamic Systems,” J. Arkansas Acad. Sci. 48, 36–40 (1994).

J. Opt. Soc. Am. A (1)

Mech. Syst. Signal. Process. (1)

K. Worden and P. L. Green, “A machine learning approach to nonlinear modal analysis,” Mech. Syst. Signal. Process. 84, 34–53 (2017).
[Crossref]

Neural Computation (1)

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation 9(8), 1735–1780 (1997).
[Crossref]

Opt. Express (2)

Opt. Photonics News (1)

D. Cornwell, “Space-Based Laser Communications Break Threshold,” Opt. Photonics News 27(5), 24–31 (2016).
[Crossref]

Other (14)

D. B. Bungbung and D. Valero, “Application of the Optical Flow Method to Velocity Determination in Hydraulic Structure Models,” in 6th International Symposium on Hydraulic Structures and Water System Management, Portland, (2016).

E. R. Davies, Computer Vision: Principles, Algorithms, Applications, Learning (Academic Press, 2017), pp. 347–357.

L. C. Andrews and R. L. Phillips, Laser Beam Propagation through Random Media (SPIE Press, 1998).

https://docs.python.org/2/library/multiprocessing.html

https://docs.python.org/3.8/library/multiprocessing.shared_memory.html

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (The MIT Press, 2016), pp. 542–550.

O. Keller, Light - The Physics of the Photon (CRC Press, 2014), pp. 378–402.

K. Cahill, Physical Mathematics (Cambridge University Press, 2013), pp. 245–256.

J. Corso, “Motion and Optical Flow,” https://web.eecs.umich.edu/∼jjcorso/t/598F14/files/lecture_1015_motion.pdf , University of Chicago (2014).

D. Margaritis, “Learning Bayesian Network Model Structure from Data,” PhD dissertation, Carnegie Mellon University, (2013)

R. C. Gonzalez and R. E. Woods, Digital Image Processing (Pearson, 2017), pp. 57–120.

J. W. Goodman, Introduction to Fourier Optics (W.H. Freeman, 2017), pp. 441–452.

P. Zarchan and H. Musoff, Fundamentals of Kalman Filtering (Progress in Aeronautics and Astronautics) (AIAA, 2015), pp. 210–212

G. Farnebäck, “Two-Frame Motion Estimation Based on Polynomial Expansion,” in Proceedings of the 13th Scandinavian Conference on Image Analysis (SCIA, 2003), pp. 363–370.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (24)

Fig. 1.
Fig. 1. Architecture of CNN.
Fig. 2.
Fig. 2. Architecture of Model PCL.
Fig. 3.
Fig. 3. Equipment for experiment.
Fig. 4.
Fig. 4. Google map for local Area.
Fig. 5.
Fig. 5. Images of Light Spots.
Fig. 6.
Fig. 6. Off-Line one time-step training and testing.
Fig. 7.
Fig. 7. Off-line training and testing for a two time-step prediction model.
Fig. 8.
Fig. 8. Two time-step training and testing curves (for the 1st time-step).
Fig. 9.
Fig. 9. Two time-step training and testing curves (for the 2nd time-step).
Fig. 10.
Fig. 10. Testing curves for the dual time-step model.
Fig. 11.
Fig. 11. Off-line training and testing for a four time-step prediction model.
Fig. 12.
Fig. 12. Four time-step training and testing curves (for the 1st time-step).
Fig. 13.
Fig. 13. Four time-step training and testing curves (for the 2nd time-step).
Fig. 14.
Fig. 14. Four time-step training and testing curves (for the 3rd time-step).
Fig. 15.
Fig. 15. Four time-step training and testing curves (for the 4th time-step).
Fig. 16.
Fig. 16. Testing errors for the four time-step training and testing model.
Fig. 17.
Fig. 17. Distributed computing for high frequency on-line process.
Fig. 18.
Fig. 18. Data arrangement for multi-process on-line training and prediction.
Fig. 19.
Fig. 19. On-line training and testing error for a single time-step model.
Fig. 20.
Fig. 20. On-line training and testing error compared with standard deviation of centroid position (500∼1000s).
Fig. 21.
Fig. 21. On-line training and testing error for a two time-step prediction model
Fig. 22.
Fig. 22. On-line training and testing error for two time-step training and prediction (750∼1500s) together with SD for centroid positions.
Fig. 23.
Fig. 23. On-line training and testing for a four time-step prediction model.
Fig. 24.
Fig. 24. On-line training and testing error for four time-step training and prediction process (3000∼4000s) together with SD for centroid positions.

Tables (1)

Tables Icon

Table 1. Equipment for on-line process.

Equations (30)

Equations on this page are rendered with MathJax. Learn more.

P [ x C ( t m + 1 ) , x C ( t m + 2 ) , , x C ( t m + k ) | I ( x ( t m ) ) , I ( x ( t m 1 ) ) , , I ( x ( t m n + 1 ) ) ]
X C = i , j ( i × I ( i , j ) ) i , j I ( i , j ) , Y C = i , j ( j × I ( i , j ) ) i , j I ( i , j ) .
X ( t m ) = F ( t m ) X ( t m 1 ) + B ( t m ) U ( t m ) + W ( t m )
R ( x , y , l , t m ) = A ( x , y , t m ) exp [ i Φ ( x , y , t m ) ] ,
Ψ ( x , y , t m ) = F x , y 1 { F ξ , η [ R ( x , y , l , t m ) ] T ( ξ , η ) }
I ( x i , y j , t m ) = S i , j [ Ψ ( x , y , 0 , t m ) Ψ ( x , y , 0 , t m ) ]
n ( x , y , 0 , t m ) = K ψ ( x , y , 0 , t m ) ψ ( x , y , 0 , t m )
I ( x i , y j , t m ) = S i , j [ n ( x , y , 0 , t m ) ]
I ( x i , y j , t m + 1 ) = I ( x i , y j , t m ) + Δ t d I ( x i , y j , t m ) d t + O I ( Δ t ) = I ( x i , y j , t m ) + Δ t I ( x i , y j , t m ) x x ˙ i + Δ t I ( x i , y j , t m ) y y ˙ j + Δ t I ( x i , y j , t m ) t + O I ( Δ t )
n ( x , y , 0 , t m + 1 ) = n ( x , y , 0 , t m ) + Δ t n ( x , y , 0 , t m ) x x ˙ + Δ t n ( x , y , 0 , t m ) y y ˙ + Δ t n ( x , y , 0 , t m ) t + O n ( Δ t ) .
I ( x i , y j , t m ) x x ˙ i = S i , j [ n ( x , y , 0 , t m ) x x ˙ ]
I ( x i , y j , t m ) y y ˙ j = S i , j [ n ( x , y , 0 , t m ) y y ˙ ]
I ( x i , y j , t m ) t = S i , j [ n ( x , y , 0 , t m ) t ]
I ( x i , y j , t m s ) x = I ( x i + 1 , y j , t m s ) I ( x i , y j , t m s ) ( i < s x 1 )
I ( x i , y j , t m s ) y = I ( x i , y j + 1 , t m s ) I ( x i , y j , t m s ) ( y < s y 1 )
I ( x i , y j , t m s ) t = I ( x i , y j , t m s ) I ( x i , y j , t m s 1 )
[ V x , V y ] = V D O F [ I ( t m s ) , I ( t m s 1 ) ]
U 4 ( t m s + 1 ) = C N N [ I ( t m s ) x , I ( t m s ) y , I ( t m s ) t , V x ( t m s ) , V y ( t m s ) , I ( t m s ) ]
F 4 n ( t m , t m 1 , , t m n + 1 ) = r e s h a p e [ U 4 T ( t m ) , U 4 T ( t m 1 ) , , U 4 T ( t m n + 1 ) ] ,
f s = σ g [ W f x s + U f h s 1 + b f ] i s = σ g ( W i x s + U i h s 1 + b i ) o s = σ g ( W o x s + U o h s 1 + b o ) c s = f s c s 1 + i s σ c ( W c x s + U c h s 1 + b c ) h s = o s σ h ( c s )
Q ( t m + k , t m + k 1 , , t m + 1 ) = L S T M [ F 4 n ( t m , t m 1 , , t m n + 1 ) ] ,
Q ( t m + k , t m + k 1 , , t m + 1 ) = [ X ( t m + k ) , X ( t m + k 1 ) , , X ( t m + 1 ) ] ,
P [ Q ( t m + k , t m + k 1 , , t m + 1 ) | I ( t m ) , I ( t m 1 ) , I ( t m 2 ) , , I ( t m n ) ] = P [ Q ( t m + k , t m + k 1 , , t m + 1 ) | U 4 ( t m + 1 ) , U 4 ( t m ) , U 4 ( t m 1 ) , , U 4 ( t m n + 2 ) ] P [ U 4 ( t m + 1 ) , U 4 ( t m ) , U 4 ( t m 1 ) , , U 4 ( t m n + 2 ) | I ( t m ) , I ( t m 1 ) , I ( t m 2 ) , , I ( t m n ) ] = P [ Q ( t m + k , t m + k 1 , , t m + 1 ) | U 4 ( t m + 1 ) , U 4 ( t m ) , U 4 ( t m 1 ) , , U 4 ( t m n + 2 ) ] P [ U 4 ( t m + 1 ) , U 4 ( t m ) , U 4 ( t m 1 ) , , U 4 ( t m n + 2 ) | ( I ( t m ) , I ( t m 1 ) ) , ( I ( t m 1 ) , I ( t m 2 ) ) , , ( I ( t m n + 1 ) , I ( t m n ) ) ] = P [ Q ( t m + k , t m + k 1 , , t m + 1 ) | U 4 ( t m + 1 ) , U 4 ( t m ) , U 4 ( t m 1 ) , , U 4 ( t m n + 2 ) ] P [ U 4 ( t m + 1 ) | ( I ( t m ) , I ( t m 1 ) ) ] P [ U 4 ( t m ) | ( I ( t m 1 ) , I ( t m 2 ) ) ] P [ U 4 ( t m n + 2 ) | ( I ( t m n + 1 ) , I ( t m n ) ) ] ,
L 1 = i b a t c h [ ( x i , 1 X i , 1 ) 2 + ( y i , 1 Y i , 1 ) 2 ]
L 2 = i b a t c h [ ( x i , 2 X i , 2 ) 2 + ( y i , 2 Y i , 2 ) 2 ]
L m = i b a t c h [ ( x i , m X i , m ) 2 + ( y i , m Y i , m ) 2 ]
L o s s = n = 1 m L n + i , j = 1 i j m ( L i L j ) 2 .
L b i n d i n g = i , j = 1 i j m ( L i L j ) 2 .
α = ( p i x e l s i z e ) ( F o c a l L e n g t h ) = 10.3 μ r a d
β = m α

Metrics