Abstract

We present a new, robust and automated method for registering sequences of images acquired from scanning ophthalmoscopes. The method uses a multi-scale B-spline representation of the deformation field to map images to each other and an hierarchical optimization method. We applied the method to video sequences acquired from different parts of the retina. In all cases, the registration was successful, even in the presence of large distortions from microsaccades, and the resulting deformation fields describe the fixational motion of the eye. The registration reveals examples of dynamic photoreceptor behaviour in the sequences.

© 2011 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. Stetter, R. A. Sendtner, and G. T. Timberlake, "A novel method for measuring saccade profiles using the scanning laser ophthalmoscope," Vision Res. 36, 1987-1994 (1996).
    [PubMed]
  2. A. Can, C. V. Stewart, B. Roysam, and H. L. Tannenbaum, "A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina," IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 347-364 (2002).
  3. S. Martinez-Conde, S. L. Macknik, and D. H. Hube, "The role of fixational eye movements in visual perception," Nat. Rev. Neurosci. 5, 229-240 (2004).
    [PubMed]
  4. J. B. Mulligan, "Recovery of motion parameters from distortions in scanned images," Proceedings of the NASA Image Registration Workshop (IRW97), NASA Goddard Space Flight Center, MD, 1997.
  5. S. B. Stevenson, and A. Roorda, "Correcting for miniature eye movements in high resolution scanning laser ophthalmoscopy," in Ophthalmic Technologies XV, edited by Fabrice Manns, Per Soderberg, Arthur Ho, Proc. of SPIE Vol. 5688A, pp. 145-151 (2005).
  6. C. R. Vogel, D. W. Arathorn, A. Roorda, and A. Parker, "Retinal motion estimation in adaptive optics scanning laser ophthalmoscopy," Opt. Express 14(2), 487-497 (2006).
    [PubMed]
  7. D. W. Arathorn, Q. Yang, C. R. Vogel, Y. Zhang, P. Tiruveedhula, and A. Roorda, "Retinally stabilized conetargeted stimulus delivery," Opt. Express 15(21), 13731-13744 (2007).
    [PubMed]
  8. H. Li, J. Lu, G. Shi, and Y. Zhang, "Tracking features in retinal images of adaptive optics confocal scanning laser ophthalmoscope using KLT-SIFT algorithm," Biomed. Opt. Express 1, 31-40 (2010).
  9. V. Nourrit, J. M. Bueno, B. Vohnsen, and P. Artal, "Nonlinear registration for scanned retinal images: application to ocular polarimetry," Appl. Opt. 47(29), 5341-5347 (2008).
    [PubMed]
  10. V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, "3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization," IEEE Trans. Image Process. 14, 553-566 (2005).
    [PubMed]
  11. J. Kybic, and M. Unser, "Fast parametric elastic image registration," IEEE Trans. Image Process. 12, 1427-1442 (2003).
  12. H. Lester, and S. R. Arridge, "A survey of hierarchical non-linear medical image registration," Pattern Recognit. 32(1), 129-149 (1999).
  13. V. Chvátal, Linear programming, (Freeman, 1983).
  14. R. S. Jonnal, J. Rha, Y. Zhang, B. Cense, W. Gao, and D. T. Miller, "In vivo functional imaging of human cone photoreceptors," Opt. Express 15(24), 16141-16160 (2007).
    [PubMed]

2010

2008

2007

2006

2005

V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, "3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization," IEEE Trans. Image Process. 14, 553-566 (2005).
[PubMed]

2004

S. Martinez-Conde, S. L. Macknik, and D. H. Hube, "The role of fixational eye movements in visual perception," Nat. Rev. Neurosci. 5, 229-240 (2004).
[PubMed]

2003

J. Kybic, and M. Unser, "Fast parametric elastic image registration," IEEE Trans. Image Process. 12, 1427-1442 (2003).

2002

A. Can, C. V. Stewart, B. Roysam, and H. L. Tannenbaum, "A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina," IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 347-364 (2002).

1999

H. Lester, and S. R. Arridge, "A survey of hierarchical non-linear medical image registration," Pattern Recognit. 32(1), 129-149 (1999).

1996

M. Stetter, R. A. Sendtner, and G. T. Timberlake, "A novel method for measuring saccade profiles using the scanning laser ophthalmoscope," Vision Res. 36, 1987-1994 (1996).
[PubMed]

Arathorn, D. W.

Armspach, J.-P.

V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, "3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization," IEEE Trans. Image Process. 14, 553-566 (2005).
[PubMed]

Arridge, S. R.

H. Lester, and S. R. Arridge, "A survey of hierarchical non-linear medical image registration," Pattern Recognit. 32(1), 129-149 (1999).

Artal, P.

Bueno, J. M.

Can, A.

A. Can, C. V. Stewart, B. Roysam, and H. L. Tannenbaum, "A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina," IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 347-364 (2002).

Cense, B.

Gao, W.

Heinrich, C.

V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, "3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization," IEEE Trans. Image Process. 14, 553-566 (2005).
[PubMed]

Heitz, F.

V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, "3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization," IEEE Trans. Image Process. 14, 553-566 (2005).
[PubMed]

Hube, D. H.

S. Martinez-Conde, S. L. Macknik, and D. H. Hube, "The role of fixational eye movements in visual perception," Nat. Rev. Neurosci. 5, 229-240 (2004).
[PubMed]

Jonnal, R. S.

Kybic, J.

J. Kybic, and M. Unser, "Fast parametric elastic image registration," IEEE Trans. Image Process. 12, 1427-1442 (2003).

Lester, H.

H. Lester, and S. R. Arridge, "A survey of hierarchical non-linear medical image registration," Pattern Recognit. 32(1), 129-149 (1999).

Li, H.

Lu, J.

Macknik, S. L.

S. Martinez-Conde, S. L. Macknik, and D. H. Hube, "The role of fixational eye movements in visual perception," Nat. Rev. Neurosci. 5, 229-240 (2004).
[PubMed]

Martinez-Conde, S.

S. Martinez-Conde, S. L. Macknik, and D. H. Hube, "The role of fixational eye movements in visual perception," Nat. Rev. Neurosci. 5, 229-240 (2004).
[PubMed]

Miller, D. T.

Noblet, V.

V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, "3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization," IEEE Trans. Image Process. 14, 553-566 (2005).
[PubMed]

Nourrit, V.

Parker, A.

Rha, J.

Roorda, A.

Roysam, B.

A. Can, C. V. Stewart, B. Roysam, and H. L. Tannenbaum, "A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina," IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 347-364 (2002).

Sendtner, R. A.

M. Stetter, R. A. Sendtner, and G. T. Timberlake, "A novel method for measuring saccade profiles using the scanning laser ophthalmoscope," Vision Res. 36, 1987-1994 (1996).
[PubMed]

Shi, G.

Stetter, M.

M. Stetter, R. A. Sendtner, and G. T. Timberlake, "A novel method for measuring saccade profiles using the scanning laser ophthalmoscope," Vision Res. 36, 1987-1994 (1996).
[PubMed]

Stewart, C. V.

A. Can, C. V. Stewart, B. Roysam, and H. L. Tannenbaum, "A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina," IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 347-364 (2002).

Tannenbaum, H. L.

A. Can, C. V. Stewart, B. Roysam, and H. L. Tannenbaum, "A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina," IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 347-364 (2002).

Timberlake, G. T.

M. Stetter, R. A. Sendtner, and G. T. Timberlake, "A novel method for measuring saccade profiles using the scanning laser ophthalmoscope," Vision Res. 36, 1987-1994 (1996).
[PubMed]

Tiruveedhula, P.

Unser, M.

J. Kybic, and M. Unser, "Fast parametric elastic image registration," IEEE Trans. Image Process. 12, 1427-1442 (2003).

Vogel, C. R.

Vohnsen, B.

Yang, Q.

Zhang, Y.

Appl. Opt.

Biomed. Opt. Express

IEEE Trans. Image Process.

V. Noblet, C. Heinrich, F. Heitz, and J.-P. Armspach, "3-D deformable image registration: a topology preservation scheme based on hierarchical deformation models and interval analysis optimization," IEEE Trans. Image Process. 14, 553-566 (2005).
[PubMed]

J. Kybic, and M. Unser, "Fast parametric elastic image registration," IEEE Trans. Image Process. 12, 1427-1442 (2003).

IEEE Trans. Pattern Anal. Mach. Intell.

A. Can, C. V. Stewart, B. Roysam, and H. L. Tannenbaum, "A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina," IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 347-364 (2002).

Nat. Rev. Neurosci.

S. Martinez-Conde, S. L. Macknik, and D. H. Hube, "The role of fixational eye movements in visual perception," Nat. Rev. Neurosci. 5, 229-240 (2004).
[PubMed]

Opt. Express

Pattern Recognit.

H. Lester, and S. R. Arridge, "A survey of hierarchical non-linear medical image registration," Pattern Recognit. 32(1), 129-149 (1999).

Vision Res.

M. Stetter, R. A. Sendtner, and G. T. Timberlake, "A novel method for measuring saccade profiles using the scanning laser ophthalmoscope," Vision Res. 36, 1987-1994 (1996).
[PubMed]

Other

V. Chvátal, Linear programming, (Freeman, 1983).

J. B. Mulligan, "Recovery of motion parameters from distortions in scanned images," Proceedings of the NASA Image Registration Workshop (IRW97), NASA Goddard Space Flight Center, MD, 1997.

S. B. Stevenson, and A. Roorda, "Correcting for miniature eye movements in high resolution scanning laser ophthalmoscopy," in Ophthalmic Technologies XV, edited by Fabrice Manns, Per Soderberg, Arthur Ho, Proc. of SPIE Vol. 5688A, pp. 145-151 (2005).

Supplementary Material (7)

» Media 1: AVI (654 KB)     
» Media 2: AVI (619 KB)     
» Media 3: AVI (654 KB)     
» Media 4: AVI (631 KB)     
» Media 5: AVI (654 KB)     
» Media 6: AVI (625 KB)     
» Media 7: AVI (47 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1

The mapping h and the associated displacement vector field u. A horizontal line in the reference is mapped by rigid translation to a shifted horizontal line in the floating image coordinates.

Fig. 2
Fig. 2

Modelling with one dimensional B-splines of order 1: (a) the triangle basis function ϕ i l ( . ), (b) the three basis functions used at scale l = 1, and (c) an example of a function which can be modelled at scale l = 1.

Fig. 3
Fig. 3

Sample images from three SLO video sequences of the retina (all 1 degree field): ∼1.5 deg eccentricity (a) raw sequence ( Media 1) and (d) stabilized sequence ( Media 2); centre of the fovea (c) raw ( Media 3) and (d) stabilized ( Media 4); and ∼ 2 deg eccentricity (e) raw ( Media 5) and (f) stabilized ( Media 6). These correspond respectively to sequences 1, 14 and 25 in Fig. 5 (Note the black regions in the registered sequences correspond to regions of the reference image that were not contained in the floating image due to the eye motion).

Fig. 4
Fig. 4

Enlarged sections showing temporal averages of the registered sequences from Fig. 3. Left: (a), (c) and (e) show the reference frames from the sequences in Fig. 3. Right: (b), (d) and (f) show the temporal averages of the registered sequences.

Fig. 5
Fig. 5

Standard deviation of irradiance in the temporal average normalized to the standard deviation in the reference image for the 26 registered video sequences. Results are shown both for sequences registered with (blue) and without (red) regularization and for the unregistered sequences (black) for comparison.

Fig. 6
Fig. 6

Displacement fields during the microsaccade motion of sequence Fig. 3 (b) calculated both with and without regularization and the difference between the two. (Note, all units are in pixels and the fields are plotted only for the valid region where the reference and registered images overlap.)

Fig. 7
Fig. 7

Image registration with microsaccade motion. Floating, reference, and the registered floating image Ifloath are shown. The plots indicate the x and y components of the registration displacement field in pixel units for the overlap region. Media 7 shows alternately the reference and registered images to illustrate the alignment.

Tables (2)

Tables Icon

Algorithm 1 Blockwise optimization procedure

Tables Icon

Algorithm 2 Whole optimization procedure at scale l

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

I reg ( s ) = I float h ( s ) I float ( h ( s ) ) .
θ opt = arg min θ Θ E ( I ref , I float h θ ) ,
h ( s ) = s + u ( s ) ,
u ( s ) = u ( s x , s y ) = u ( s x ) = [ u x ( s x ) , u y ( s x ) ] T ,
u l ( s x ) = ( u x l ( s x ) u y l ( s x ) ) = ( i a x ; i l ϕ i l ( s x ) i a y ; i l ϕ i l ( s x ) ) , with ϕ i l ( x ) = 2 l / 2 ϕ ( 2 l x i )
E ( sim ) ( h ) = 1 N ( h ) 1 s Ω d ; h ( s ) Ω f ( I ref ( s ) I float h ( s ) ) ,
E ( reg ) ( h ) = Ω x ( u x ( x ) x ) 2 + ( u y ( x ) x ) 2 d x .
E = ( 1 λ ) E ( sim ) + λ C E ( reg ) ,
a x ; i l a x ; i 1 l > 1 2 3 l / 2 and a x ; i l a x ; i + 1 l < 1 2 3 l / 2 .

Metrics