Abstract

This paper presents a computationally efficient method for the measurement of a dense image correspondence vector field using supplementary data from an inertial navigation sensor (INS). The application is suited to airborne imaging systems, such as an unmanned air vehicle, where size, weight, and power restrictions limit the amount of onboard processing available. The limited processing will typically exclude the use of traditional, but computationally expensive, optical flow and block matching algorithms, such as Lucas–Kanade, Horn–Schunck, or the adaptive rood pattern search. Alternatively, the measurements obtained from an INS, on board the platform, lead to a closed-form solution to the correspondence field. Airborne platforms are well suited to this application because they already possess INSs and global positioning systems as part of their existing avionics package. We derive the closed-form solution for the image correspondence vector field based on the INS data. We then show, through both simulations and real flight data, that the closed-form inertial sensor solution outperforms traditional optical flow and block matching methods.

© 2012 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. R. Schultz and L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996–1011 (1996).
    [CrossRef]
  2. K. Katsaggelos, R. Molina, and J. Mateos, Super Resolution of Images and Video (Morgan & Claypool, 2007).
  3. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI'81) (IJCAI, 1981), pp. 674–679.
  4. L. Zhang, Q. Yuan, H. Shen, and P. Li, “Multiframe image super-resolution adapted with local spatial information,” J. Opt. Soc. Am. A 28, 381–390 (2011).
    [CrossRef]
  5. S. S. Mokri, A. Hussain, N. Ibrahim, and M. M. Mustafa, “Motion detection using Lucas–Kanade algorithm and application enhancement,” in 2009 International Conference on Electrical Engineering and Informatics (ICEEI’09) (ICEEI, 2009), Vol. 2, pp. 537–542.
  6. K. P. Horn and G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
    [CrossRef]
  7. S. Zhu and K. Ma, “A new diamond search algorithm for fast block matching motion estimation,” IEEE Trans. Image Process. 9, 287–290 (2000).
    [CrossRef]
  8. Y. Nie and K.-K. Ma, “Adaptive rood pattern search for fast block-matching motion estimation,” IEEE Trans. Image Process. 11, 1442–1449 (2002).
    [CrossRef]
  9. D. Gosgen-Neskin and I. Y. Bar-Itzhack, “Unified approach to inertial navigation system error modeling,” J. Guid. Control Dyn. 15, 648–653 (1992).
    [CrossRef]
  10. C. Hide, T. Moore, and M. Smith, “Adaptive Kalman filtering algorithms for integrating GPS and low cost INS,” in Proceedings of the IEEE Position, Location, and Navigation Symposium (IEEE, 2004), pp. 227–233.
  11. I. Rhee, M. F. Abdel-Hafez, and J. L. Speyer, “Observability of an integrated GPS/INS during maneuvers,” IEEE Trans. Aerosp. Electron. Syst. 40, 526–535 (2004).
    [CrossRef]
  12. D. Gebre-Egziabher, R. C. Hayward, and J. D. Powell, “A low-cost GPS/inertial attitude heading reference system (AHRS) for general aviation applications,” in Proceedings of the IEEE Position, Location, and Navigation Symposium, (IEEE, 1998), pp. 518–525.
  13. D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, 2003).
  14. Z. Zhang, “A flexible new technique for camera calibration,” Tech. Rep. MSR-TR-98-71 (Microsoft, 2008).
  15. B. J. Lei, E. A. Hendriks, and A. K. Katsaggelos, “Camera calibration for 3D reconstruction and view transformation,” in 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body, N. Sarris and M. G. Strintzis, eds. (IRM, 2005), pp. 70–129.
  16. “Performance specification digital terrain elevation data (DTED),” MIL-PRF-89020B (National Information Services, May2000).
  17. K. Chen, G. Zhao, Z. Meng, J. Yan, and H. Lu, “Equivalent approaches to equations of traditional transfer alignment and rapid transfer alignment,” in Proceedings of the 7th World Congress on Intelligent Control and Automation (2008), pp. 892–895.
  18. L. Joon and L. You-Chol, “Transfer alignment considering measurement time delay and ship body flexure,” J. Mech. Sci. Tech. 23, 195–203 (2009).
    [CrossRef]
  19. J. Shortelle, R. Graham, and C. Rabourn, “F-16 Flight test of a rapid transfer alignment procedure,” presented at the IEEE Position, Location, and Navigation Symposium, Palm Springs, California, 20–23 April 1998.
  20. M. Veth and J. Raquet, “Alignment and calibration of optical and inertial sensors using stellar observations” (Air Force Institute of Technology, 2007).
  21. G. Fasano, D. Accardo, A. Moccia, and A. Rispoli, “An innovative procedure for calibration of strapdown electro-optical sensors onboard unmanned air vehicles,” Sensors 10, 639–654 (2010).
    [CrossRef]
  22. J. Shi and C. Tomasi, “Good features to track,” in 1994 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’94) (IEEE Computer Society, 1994), pp. 593–600.
  23. C. Tomasi and T. Kanade, “Shape and motion from image streams: a factorization method: full report on the orthographic case,” CMU Tech. Rep. CMU-CS-92-104, March1992.
  24. Silver Fox, http://www.satnews.com/cgi-bin/story.cgi?number=1554529420 .
  25. L. Stevens and L. Lewis, Aircraft Control and Simulation(Wiley, 1992).
  26. H. Blakelock, Automatic Control of Aircraft and Missiles(Wiley, 1991).
  27. I. Lizarraga, “Autonomous landing system for a UAV,” Master’s thesis (Naval Post Graduate School, 2004).
  28. Google Earth, http://earth.google.com .
  29. J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).
  30. L. Serrano, D. Kim, and R. B. Langley, “A GPS velocity sensor: how accurate can it be?—A first look,” presented at ION NTM 2004,San Diego, California, 26–28 January 2004.
  31. S. Khan, Optical Flow MATLAB Code, % http://www.cs.ucf.edu/vision/public_html/source.html#Optical%20Flow .
  32. MATLAB Central, http://www.mathworks.com/matlabcentral/fileexchange/8761-block-matching-algorithms-for-motion-estimation.
  33. L.-M. Po and W.-C. Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol. 6, 313–317 (1996).
    [CrossRef]
  34. M. Chi, D. Tran, and Etienne-Cummings, “Optical flow approximation of sub-pixel accurate block matching,” in IEEE Conference on Acoustics, Speech and Signal Processing (IEEE, 2007), pp. I-1017–I-1020.
  35. H. Chan, T. Vo, and Q. Nguyen, “Subpixel motion estimation without interpolation,” in IEEE Conference Acoustics, Speech and Signal Processing (IEEE, 2010), pp. 722–725.
  36. FLIR Systems: http://www.flir.com/US .

2011 (1)

2010 (1)

G. Fasano, D. Accardo, A. Moccia, and A. Rispoli, “An innovative procedure for calibration of strapdown electro-optical sensors onboard unmanned air vehicles,” Sensors 10, 639–654 (2010).
[CrossRef]

2009 (1)

L. Joon and L. You-Chol, “Transfer alignment considering measurement time delay and ship body flexure,” J. Mech. Sci. Tech. 23, 195–203 (2009).
[CrossRef]

2004 (1)

I. Rhee, M. F. Abdel-Hafez, and J. L. Speyer, “Observability of an integrated GPS/INS during maneuvers,” IEEE Trans. Aerosp. Electron. Syst. 40, 526–535 (2004).
[CrossRef]

2002 (1)

Y. Nie and K.-K. Ma, “Adaptive rood pattern search for fast block-matching motion estimation,” IEEE Trans. Image Process. 11, 1442–1449 (2002).
[CrossRef]

2000 (1)

S. Zhu and K. Ma, “A new diamond search algorithm for fast block matching motion estimation,” IEEE Trans. Image Process. 9, 287–290 (2000).
[CrossRef]

1997 (1)

J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).

1996 (2)

L.-M. Po and W.-C. Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol. 6, 313–317 (1996).
[CrossRef]

R. Schultz and L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996–1011 (1996).
[CrossRef]

1992 (1)

D. Gosgen-Neskin and I. Y. Bar-Itzhack, “Unified approach to inertial navigation system error modeling,” J. Guid. Control Dyn. 15, 648–653 (1992).
[CrossRef]

1981 (1)

K. P. Horn and G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

Abdel-Hafez, M. F.

I. Rhee, M. F. Abdel-Hafez, and J. L. Speyer, “Observability of an integrated GPS/INS during maneuvers,” IEEE Trans. Aerosp. Electron. Syst. 40, 526–535 (2004).
[CrossRef]

Accardo, D.

G. Fasano, D. Accardo, A. Moccia, and A. Rispoli, “An innovative procedure for calibration of strapdown electro-optical sensors onboard unmanned air vehicles,” Sensors 10, 639–654 (2010).
[CrossRef]

Bar-Itzhack, I. Y.

D. Gosgen-Neskin and I. Y. Bar-Itzhack, “Unified approach to inertial navigation system error modeling,” J. Guid. Control Dyn. 15, 648–653 (1992).
[CrossRef]

Blakelock, H.

H. Blakelock, Automatic Control of Aircraft and Missiles(Wiley, 1991).

Cannon, M. E.

J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).

Chan, H.

H. Chan, T. Vo, and Q. Nguyen, “Subpixel motion estimation without interpolation,” in IEEE Conference Acoustics, Speech and Signal Processing (IEEE, 2010), pp. 722–725.

Chen, K.

K. Chen, G. Zhao, Z. Meng, J. Yan, and H. Lu, “Equivalent approaches to equations of traditional transfer alignment and rapid transfer alignment,” in Proceedings of the 7th World Congress on Intelligent Control and Automation (2008), pp. 892–895.

Chi, M.

M. Chi, D. Tran, and Etienne-Cummings, “Optical flow approximation of sub-pixel accurate block matching,” in IEEE Conference on Acoustics, Speech and Signal Processing (IEEE, 2007), pp. I-1017–I-1020.

Etienne-Cummings,

M. Chi, D. Tran, and Etienne-Cummings, “Optical flow approximation of sub-pixel accurate block matching,” in IEEE Conference on Acoustics, Speech and Signal Processing (IEEE, 2007), pp. I-1017–I-1020.

Fasano, G.

G. Fasano, D. Accardo, A. Moccia, and A. Rispoli, “An innovative procedure for calibration of strapdown electro-optical sensors onboard unmanned air vehicles,” Sensors 10, 639–654 (2010).
[CrossRef]

Forsyth, D. A.

D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, 2003).

Gebre-Egziabher, D.

D. Gebre-Egziabher, R. C. Hayward, and J. D. Powell, “A low-cost GPS/inertial attitude heading reference system (AHRS) for general aviation applications,” in Proceedings of the IEEE Position, Location, and Navigation Symposium, (IEEE, 1998), pp. 518–525.

Gosgen-Neskin, D.

D. Gosgen-Neskin and I. Y. Bar-Itzhack, “Unified approach to inertial navigation system error modeling,” J. Guid. Control Dyn. 15, 648–653 (1992).
[CrossRef]

Graham, R.

J. Shortelle, R. Graham, and C. Rabourn, “F-16 Flight test of a rapid transfer alignment procedure,” presented at the IEEE Position, Location, and Navigation Symposium, Palm Springs, California, 20–23 April 1998.

Hayward, R. C.

D. Gebre-Egziabher, R. C. Hayward, and J. D. Powell, “A low-cost GPS/inertial attitude heading reference system (AHRS) for general aviation applications,” in Proceedings of the IEEE Position, Location, and Navigation Symposium, (IEEE, 1998), pp. 518–525.

Hendriks, E. A.

B. J. Lei, E. A. Hendriks, and A. K. Katsaggelos, “Camera calibration for 3D reconstruction and view transformation,” in 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body, N. Sarris and M. G. Strintzis, eds. (IRM, 2005), pp. 70–129.

Herbert, J. M.

J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).

Hide, C.

C. Hide, T. Moore, and M. Smith, “Adaptive Kalman filtering algorithms for integrating GPS and low cost INS,” in Proceedings of the IEEE Position, Location, and Navigation Symposium (IEEE, 2004), pp. 227–233.

Horn, K. P.

K. P. Horn and G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

Hussain, A.

S. S. Mokri, A. Hussain, N. Ibrahim, and M. M. Mustafa, “Motion detection using Lucas–Kanade algorithm and application enhancement,” in 2009 International Conference on Electrical Engineering and Informatics (ICEEI’09) (ICEEI, 2009), Vol. 2, pp. 537–542.

Ibrahim, N.

S. S. Mokri, A. Hussain, N. Ibrahim, and M. M. Mustafa, “Motion detection using Lucas–Kanade algorithm and application enhancement,” in 2009 International Conference on Electrical Engineering and Informatics (ICEEI’09) (ICEEI, 2009), Vol. 2, pp. 537–542.

Jokerst, S.

J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).

Joon, L.

L. Joon and L. You-Chol, “Transfer alignment considering measurement time delay and ship body flexure,” J. Mech. Sci. Tech. 23, 195–203 (2009).
[CrossRef]

Kanade, T.

C. Tomasi and T. Kanade, “Shape and motion from image streams: a factorization method: full report on the orthographic case,” CMU Tech. Rep. CMU-CS-92-104, March1992.

D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI'81) (IJCAI, 1981), pp. 674–679.

Katsaggelos, A. K.

B. J. Lei, E. A. Hendriks, and A. K. Katsaggelos, “Camera calibration for 3D reconstruction and view transformation,” in 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body, N. Sarris and M. G. Strintzis, eds. (IRM, 2005), pp. 70–129.

Katsaggelos, K.

K. Katsaggelos, R. Molina, and J. Mateos, Super Resolution of Images and Video (Morgan & Claypool, 2007).

Keith, J.

J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).

Kim, D.

L. Serrano, D. Kim, and R. B. Langley, “A GPS velocity sensor: how accurate can it be?—A first look,” presented at ION NTM 2004,San Diego, California, 26–28 January 2004.

Lachapelle, G.

J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).

Langley, R. B.

L. Serrano, D. Kim, and R. B. Langley, “A GPS velocity sensor: how accurate can it be?—A first look,” presented at ION NTM 2004,San Diego, California, 26–28 January 2004.

Lei, B. J.

B. J. Lei, E. A. Hendriks, and A. K. Katsaggelos, “Camera calibration for 3D reconstruction and view transformation,” in 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body, N. Sarris and M. G. Strintzis, eds. (IRM, 2005), pp. 70–129.

Lewis, L.

L. Stevens and L. Lewis, Aircraft Control and Simulation(Wiley, 1992).

Li, P.

Lizarraga, I.

I. Lizarraga, “Autonomous landing system for a UAV,” Master’s thesis (Naval Post Graduate School, 2004).

Lu, H.

K. Chen, G. Zhao, Z. Meng, J. Yan, and H. Lu, “Equivalent approaches to equations of traditional transfer alignment and rapid transfer alignment,” in Proceedings of the 7th World Congress on Intelligent Control and Automation (2008), pp. 892–895.

Lucas, D.

D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI'81) (IJCAI, 1981), pp. 674–679.

Ma, K.

S. Zhu and K. Ma, “A new diamond search algorithm for fast block matching motion estimation,” IEEE Trans. Image Process. 9, 287–290 (2000).
[CrossRef]

Ma, K.-K.

Y. Nie and K.-K. Ma, “Adaptive rood pattern search for fast block-matching motion estimation,” IEEE Trans. Image Process. 11, 1442–1449 (2002).
[CrossRef]

Ma, W.-C.

L.-M. Po and W.-C. Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol. 6, 313–317 (1996).
[CrossRef]

Mateos, J.

K. Katsaggelos, R. Molina, and J. Mateos, Super Resolution of Images and Video (Morgan & Claypool, 2007).

Meng, Z.

K. Chen, G. Zhao, Z. Meng, J. Yan, and H. Lu, “Equivalent approaches to equations of traditional transfer alignment and rapid transfer alignment,” in Proceedings of the 7th World Congress on Intelligent Control and Automation (2008), pp. 892–895.

Moccia, A.

G. Fasano, D. Accardo, A. Moccia, and A. Rispoli, “An innovative procedure for calibration of strapdown electro-optical sensors onboard unmanned air vehicles,” Sensors 10, 639–654 (2010).
[CrossRef]

Mokri, S. S.

S. S. Mokri, A. Hussain, N. Ibrahim, and M. M. Mustafa, “Motion detection using Lucas–Kanade algorithm and application enhancement,” in 2009 International Conference on Electrical Engineering and Informatics (ICEEI’09) (ICEEI, 2009), Vol. 2, pp. 537–542.

Molina, R.

K. Katsaggelos, R. Molina, and J. Mateos, Super Resolution of Images and Video (Morgan & Claypool, 2007).

Moore, T.

C. Hide, T. Moore, and M. Smith, “Adaptive Kalman filtering algorithms for integrating GPS and low cost INS,” in Proceedings of the IEEE Position, Location, and Navigation Symposium (IEEE, 2004), pp. 227–233.

Mustafa, M. M.

S. S. Mokri, A. Hussain, N. Ibrahim, and M. M. Mustafa, “Motion detection using Lucas–Kanade algorithm and application enhancement,” in 2009 International Conference on Electrical Engineering and Informatics (ICEEI’09) (ICEEI, 2009), Vol. 2, pp. 537–542.

Nguyen, Q.

H. Chan, T. Vo, and Q. Nguyen, “Subpixel motion estimation without interpolation,” in IEEE Conference Acoustics, Speech and Signal Processing (IEEE, 2010), pp. 722–725.

Nie, Y.

Y. Nie and K.-K. Ma, “Adaptive rood pattern search for fast block-matching motion estimation,” IEEE Trans. Image Process. 11, 1442–1449 (2002).
[CrossRef]

Po, L.-M.

L.-M. Po and W.-C. Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol. 6, 313–317 (1996).
[CrossRef]

Ponce, J.

D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, 2003).

Powell, J. D.

D. Gebre-Egziabher, R. C. Hayward, and J. D. Powell, “A low-cost GPS/inertial attitude heading reference system (AHRS) for general aviation applications,” in Proceedings of the IEEE Position, Location, and Navigation Symposium, (IEEE, 1998), pp. 518–525.

Rabourn, C.

J. Shortelle, R. Graham, and C. Rabourn, “F-16 Flight test of a rapid transfer alignment procedure,” presented at the IEEE Position, Location, and Navigation Symposium, Palm Springs, California, 20–23 April 1998.

Raquet, J.

M. Veth and J. Raquet, “Alignment and calibration of optical and inertial sensors using stellar observations” (Air Force Institute of Technology, 2007).

Rhee, I.

I. Rhee, M. F. Abdel-Hafez, and J. L. Speyer, “Observability of an integrated GPS/INS during maneuvers,” IEEE Trans. Aerosp. Electron. Syst. 40, 526–535 (2004).
[CrossRef]

Rispoli, A.

G. Fasano, D. Accardo, A. Moccia, and A. Rispoli, “An innovative procedure for calibration of strapdown electro-optical sensors onboard unmanned air vehicles,” Sensors 10, 639–654 (2010).
[CrossRef]

Ryan, S.

J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).

Schultz, R.

R. Schultz and L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996–1011 (1996).
[CrossRef]

Schunck, G.

K. P. Horn and G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

Serrano, L.

L. Serrano, D. Kim, and R. B. Langley, “A GPS velocity sensor: how accurate can it be?—A first look,” presented at ION NTM 2004,San Diego, California, 26–28 January 2004.

Shen, H.

Shi, J.

J. Shi and C. Tomasi, “Good features to track,” in 1994 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’94) (IEEE Computer Society, 1994), pp. 593–600.

Shortelle, J.

J. Shortelle, R. Graham, and C. Rabourn, “F-16 Flight test of a rapid transfer alignment procedure,” presented at the IEEE Position, Location, and Navigation Symposium, Palm Springs, California, 20–23 April 1998.

Smith, M.

C. Hide, T. Moore, and M. Smith, “Adaptive Kalman filtering algorithms for integrating GPS and low cost INS,” in Proceedings of the IEEE Position, Location, and Navigation Symposium (IEEE, 2004), pp. 227–233.

Speyer, J. L.

I. Rhee, M. F. Abdel-Hafez, and J. L. Speyer, “Observability of an integrated GPS/INS during maneuvers,” IEEE Trans. Aerosp. Electron. Syst. 40, 526–535 (2004).
[CrossRef]

Stevens, L.

L. Stevens and L. Lewis, Aircraft Control and Simulation(Wiley, 1992).

Stevenson, L.

R. Schultz and L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996–1011 (1996).
[CrossRef]

Szarmes, M. C.

J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).

Tomasi, C.

J. Shi and C. Tomasi, “Good features to track,” in 1994 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’94) (IEEE Computer Society, 1994), pp. 593–600.

C. Tomasi and T. Kanade, “Shape and motion from image streams: a factorization method: full report on the orthographic case,” CMU Tech. Rep. CMU-CS-92-104, March1992.

Tran, D.

M. Chi, D. Tran, and Etienne-Cummings, “Optical flow approximation of sub-pixel accurate block matching,” in IEEE Conference on Acoustics, Speech and Signal Processing (IEEE, 2007), pp. I-1017–I-1020.

Veth, M.

M. Veth and J. Raquet, “Alignment and calibration of optical and inertial sensors using stellar observations” (Air Force Institute of Technology, 2007).

Vo, T.

H. Chan, T. Vo, and Q. Nguyen, “Subpixel motion estimation without interpolation,” in IEEE Conference Acoustics, Speech and Signal Processing (IEEE, 2010), pp. 722–725.

Yan, J.

K. Chen, G. Zhao, Z. Meng, J. Yan, and H. Lu, “Equivalent approaches to equations of traditional transfer alignment and rapid transfer alignment,” in Proceedings of the 7th World Congress on Intelligent Control and Automation (2008), pp. 892–895.

You-Chol, L.

L. Joon and L. You-Chol, “Transfer alignment considering measurement time delay and ship body flexure,” J. Mech. Sci. Tech. 23, 195–203 (2009).
[CrossRef]

Yuan, Q.

Zhang, L.

Zhang, Z.

Z. Zhang, “A flexible new technique for camera calibration,” Tech. Rep. MSR-TR-98-71 (Microsoft, 2008).

Zhao, G.

K. Chen, G. Zhao, Z. Meng, J. Yan, and H. Lu, “Equivalent approaches to equations of traditional transfer alignment and rapid transfer alignment,” in Proceedings of the 7th World Congress on Intelligent Control and Automation (2008), pp. 892–895.

Zhu, S.

S. Zhu and K. Ma, “A new diamond search algorithm for fast block matching motion estimation,” IEEE Trans. Image Process. 9, 287–290 (2000).
[CrossRef]

Artif. Intell. (1)

K. P. Horn and G. Schunck, “Determining optical flow,” Artif. Intell. 17, 185–203 (1981).
[CrossRef]

IEEE Trans. Aerosp. Electron. Syst. (1)

I. Rhee, M. F. Abdel-Hafez, and J. L. Speyer, “Observability of an integrated GPS/INS during maneuvers,” IEEE Trans. Aerosp. Electron. Syst. 40, 526–535 (2004).
[CrossRef]

IEEE Trans. Circuits Syst. Video Technol. (1)

L.-M. Po and W.-C. Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol. 6, 313–317 (1996).
[CrossRef]

IEEE Trans. Image Process. (3)

S. Zhu and K. Ma, “A new diamond search algorithm for fast block matching motion estimation,” IEEE Trans. Image Process. 9, 287–290 (2000).
[CrossRef]

Y. Nie and K.-K. Ma, “Adaptive rood pattern search for fast block-matching motion estimation,” IEEE Trans. Image Process. 11, 1442–1449 (2002).
[CrossRef]

R. Schultz and L. Stevenson, “Extraction of high-resolution frames from video sequences,” IEEE Trans. Image Process. 5, 996–1011 (1996).
[CrossRef]

J. Guid. Control Dyn. (1)

D. Gosgen-Neskin and I. Y. Bar-Itzhack, “Unified approach to inertial navigation system error modeling,” J. Guid. Control Dyn. 15, 648–653 (1992).
[CrossRef]

J. Mech. Sci. Tech. (1)

L. Joon and L. You-Chol, “Transfer alignment considering measurement time delay and ship body flexure,” J. Mech. Sci. Tech. 23, 195–203 (2009).
[CrossRef]

J. Opt. Soc. Am. A (1)

Navigation (1)

J. M. Herbert, J. Keith, S. Ryan, G. Lachapelle, M. C. Szarmes, S. Jokerst, and M. E. Cannon, “DGPS kinematic carrier phase signal simulation analysis for precise aircraft velocity determination,” Navigation 44231–246 (1997).

Sensors (1)

G. Fasano, D. Accardo, A. Moccia, and A. Rispoli, “An innovative procedure for calibration of strapdown electro-optical sensors onboard unmanned air vehicles,” Sensors 10, 639–654 (2010).
[CrossRef]

Other (25)

J. Shi and C. Tomasi, “Good features to track,” in 1994 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’94) (IEEE Computer Society, 1994), pp. 593–600.

C. Tomasi and T. Kanade, “Shape and motion from image streams: a factorization method: full report on the orthographic case,” CMU Tech. Rep. CMU-CS-92-104, March1992.

Silver Fox, http://www.satnews.com/cgi-bin/story.cgi?number=1554529420 .

L. Stevens and L. Lewis, Aircraft Control and Simulation(Wiley, 1992).

H. Blakelock, Automatic Control of Aircraft and Missiles(Wiley, 1991).

I. Lizarraga, “Autonomous landing system for a UAV,” Master’s thesis (Naval Post Graduate School, 2004).

Google Earth, http://earth.google.com .

S. S. Mokri, A. Hussain, N. Ibrahim, and M. M. Mustafa, “Motion detection using Lucas–Kanade algorithm and application enhancement,” in 2009 International Conference on Electrical Engineering and Informatics (ICEEI’09) (ICEEI, 2009), Vol. 2, pp. 537–542.

K. Katsaggelos, R. Molina, and J. Mateos, Super Resolution of Images and Video (Morgan & Claypool, 2007).

D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI'81) (IJCAI, 1981), pp. 674–679.

C. Hide, T. Moore, and M. Smith, “Adaptive Kalman filtering algorithms for integrating GPS and low cost INS,” in Proceedings of the IEEE Position, Location, and Navigation Symposium (IEEE, 2004), pp. 227–233.

L. Serrano, D. Kim, and R. B. Langley, “A GPS velocity sensor: how accurate can it be?—A first look,” presented at ION NTM 2004,San Diego, California, 26–28 January 2004.

S. Khan, Optical Flow MATLAB Code, % http://www.cs.ucf.edu/vision/public_html/source.html#Optical%20Flow .

MATLAB Central, http://www.mathworks.com/matlabcentral/fileexchange/8761-block-matching-algorithms-for-motion-estimation.

D. Gebre-Egziabher, R. C. Hayward, and J. D. Powell, “A low-cost GPS/inertial attitude heading reference system (AHRS) for general aviation applications,” in Proceedings of the IEEE Position, Location, and Navigation Symposium, (IEEE, 1998), pp. 518–525.

D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach (Prentice-Hall, 2003).

Z. Zhang, “A flexible new technique for camera calibration,” Tech. Rep. MSR-TR-98-71 (Microsoft, 2008).

B. J. Lei, E. A. Hendriks, and A. K. Katsaggelos, “Camera calibration for 3D reconstruction and view transformation,” in 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body, N. Sarris and M. G. Strintzis, eds. (IRM, 2005), pp. 70–129.

“Performance specification digital terrain elevation data (DTED),” MIL-PRF-89020B (National Information Services, May2000).

K. Chen, G. Zhao, Z. Meng, J. Yan, and H. Lu, “Equivalent approaches to equations of traditional transfer alignment and rapid transfer alignment,” in Proceedings of the 7th World Congress on Intelligent Control and Automation (2008), pp. 892–895.

J. Shortelle, R. Graham, and C. Rabourn, “F-16 Flight test of a rapid transfer alignment procedure,” presented at the IEEE Position, Location, and Navigation Symposium, Palm Springs, California, 20–23 April 1998.

M. Veth and J. Raquet, “Alignment and calibration of optical and inertial sensors using stellar observations” (Air Force Institute of Technology, 2007).

M. Chi, D. Tran, and Etienne-Cummings, “Optical flow approximation of sub-pixel accurate block matching,” in IEEE Conference on Acoustics, Speech and Signal Processing (IEEE, 2007), pp. I-1017–I-1020.

H. Chan, T. Vo, and Q. Nguyen, “Subpixel motion estimation without interpolation,” in IEEE Conference Acoustics, Speech and Signal Processing (IEEE, 2010), pp. 722–725.

FLIR Systems: http://www.flir.com/US .

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1.
Fig. 1.

Normalized perspective projection model.

Fig. 2.
Fig. 2.

Range projection to ground.

Fig. 3.
Fig. 3.

Top-level simulation architecture.

Fig. 4.
Fig. 4.

Simulated aircraft camera imagery. (Left) Ground truth image. (Right) Projection into Silver Fox visual camera.

Fig. 5.
Fig. 5.

Rear-view illustration of an aircraft turn maneuver.

Fig. 6.
Fig. 6.

Top-level autopilot block diagram.

Fig. 7.
Fig. 7.

Aircraft maneuver during simulation.

Fig. 8.
Fig. 8.

Top–down view of simulated aircraft turn maneuver.

Fig. 9.
Fig. 9.

Quiver plot of optical flow vectors from the (top-left) inertial, (top-right) LK, and (bottom-right) adaptive rood pattern search algorithms for a pair of 100 Hz frames during the bank and turn maneuver at point B.

Fig. 10.
Fig. 10.

Overlay of the cumulative error distributions of the LK, closed-form inertial correspondence, and adaptive rood pattern search (BM) algorithms for (top) point A and (bottom) point B.

Fig. 11.
Fig. 11.

Comparison of 95th percentile correspondence errors for the LK, closed-form inertial, and adaptive rood pattern search algorithms over the duration of the simulated flight.

Fig. 12.
Fig. 12.

(Left) Real 60 deg FOV, infrared camera imagery over mountainous terrain. (Right) Collection location shown in Google Earth with a box showing the cameras FOV.

Fig. 13.
Fig. 13.

Overlay of the cumulative error distributions of the LK and BM algorithms, relative to the analytic correspondence calculation, for real camera imagery.

Fig. 14.
Fig. 14.

Local ground altitude map from DTED [Aircraft at position (0,0) when evaluation imagery collected].

Equations (26)

Equations on this page are rendered with MathJax. Learn more.

aP=[TNEDP]aNED.
[xy1]T=[x/zy/z1]T.
(u,v)=f(x,y)
(x,y)=f1(u,v).
aS=[TPS]aP.
RGNED(x,y,k)=RGNED(x,y,k)[xy1]T[TNEDP]kT[TPS]T(xy1),
ΔR=ΔtVk+1NED+VkNED2,
RGNED(x+Δx,y+Δy,k+1)=RGNED(x,y,k)ΔR.
(x+Δxy+Δy1)=1α([TPS][TNEDP]k+1RGNED(x+Δx,y+Δy,k+1)),
(ΔxΔy1)=1α([TPS][TNEDP]k+1(RGNED(x,y,k)ΔR))[xy0]T.
[TNEDP]=(Iωxτ)[T^NEDP],
[TPS]=[T^PS](Iδx),
δx=(0δ3δ2δ30δ1δ2δ10).
α[(ΔxΔy)IP(ΔxΔy)IS]=(100010)[AAω](δτ),
A=[T^PS][RGNED(x,y,k)[xy1]T((yk+1P)x[T^kk+1](ykP)x)Vk+1PΔt],
yk+1P=[T^kk+1][T^PS]T(xy1),
ykP=[T^PS]T(xy1),
[T^kk+1]=[T^NEDP]k+1[T^NEDP]kT,
Vk+1P=[T^NEDP]k+1(Vk+1NED+VkNED2).
ε=(ΔxESTΔxTRUE)2+(ΔyESTΔyTRUE)2,
[TNEDP]k[D1][TNEDP]k,
[TNEDP]k+1[D2][TNEDP]k+1,
ΔRΔR+DvΔt,
[D1]=Iπ180[0.04r10.04r20.36r3]x,
[D2]=Iπ180[0.0028r40.0028r50.00286r6]x,
Dv=(1ms)[r7r8r9]T.

Metrics