Abstract

We present a fully automated adaptive optics (AO) retinal image montaging algorithm using classic scale invariant feature transform with random sample consensus for outlier removal. Our approach is capable of using information from multiple AO modalities (confocal, split detection, and dark field) and can accurately detect discontinuities in the montage. The algorithm output is compared to manual montaging by evaluating the similarity of the overlapping regions after montaging, and calculating the detection rate of discontinuities in the montage. Our results show that the proposed algorithm has high alignment accuracy and a discontinuity detection rate that is comparable (and often superior) to manual montaging. In addition, we analyze and show the benefits of using multiple modalities in the montaging process. We provide the algorithm presented in this paper as open-source and freely available to download.

© 2016 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Fast adaptive optics scanning light ophthalmoscope retinal montaging

Benjamin Davidson, Angelos Kalitzeos, Joseph Carroll, Alfredo Dubra, Sebastien Ourselin, Michel Michaelides, and Christos Bergeles
Biomed. Opt. Express 9(9) 4317-4328 (2018)

Photoreceptor counting and montaging of en-face retinal images from an adaptive optics fundus camera

Bai Xue, Stacey S. Choi, Nathan Doble, and John S. Werner
J. Opt. Soc. Am. A 24(5) 1364-1372 (2007)

Automated identification of cone photoreceptors in adaptive optics retinal images

Kaccie Y. Li and Austin Roorda
J. Opt. Soc. Am. A 24(5) 1358-1363 (2007)

References

  • View by:
  • |
  • |
  • |

  1. J. Liang, D. R. Williams, and D. T. Miller, “Supernormal vision and high-resolution retinal imaging through adaptive optics,” J. Opt. Soc. Am. 14, 2884–2892 (1997).
    [Crossref]
  2. A. Roorda, A. B. Metha, P. Lennie, and D. R. Williams, “Packing arrangement of the three cone classes in primate retina,” Vision Res. 41, 1291–1306 (2001).
    [Crossref] [PubMed]
  3. A. Dubra, Y. Sulai, J. L. Norris, R. F. Cooper, A. M. Dubis, D. R. Williams, and J. Carroll, “Noninvasive imaging of the human rod photoreceptor mosaic using a confocal adaptive optics scanning ophthalmoscope,” Biomed. Opt. Express 2, 1864–1876 (2011).
    [Crossref] [PubMed]
  4. J. I. Morgan, A. Dubra, R. Wolfe, W. H. Merigan, and D. R. Williams, “In vivo autofluorescence imaging of the human and macaque retinal pigment epithelial cell mosaic,” Invest. Ophthalmol. Vis. Sci. 50, 1350–1359 (2009).
    [Crossref]
  5. A. Roorda, F. Romero-Borja, W. Donnelly, H. Queener, T. Hebert, and M. Campbell, “Adaptive optics scanning laser ophthalmoscopy,” Opt. Express 10, 405–412 (2002).
    [Crossref] [PubMed]
  6. T. Y. Chui, D. A. VanNasdale, and S. A. Burns, “The use of forward scatter to improve retinal vascular imaging with an adaptive optics scanning laser ophthalmoscope,” Biomed. Opt. Express 3, 2537–2549 (2012).
    [Crossref] [PubMed]
  7. J. I. Morgan, “The fundus photo has met its match: optical coherence tomography and adaptive optics ophthalmoscopy are here to stay,” Ophthalmic. Physiol. Opt. 36, 218–239 (2016).
    [Crossref] [PubMed]
  8. H. Li, J. Lu, G. Shi, and Y. Zhang, “Automatic montage of retinal images in adaptive optics confocal scanning laser ophthalmoscope,” Opt. Eng. 51, 057008 (2012).
    [Crossref]
  9. Y. Ke and R. Sukthankar, “PCA-SIFT: A more distinctive representation for local image descriptors,” in “Proc. of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition,”, vol. 2 (IEEE, 2004), vol. 2, pp. II–506.
  10. D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
    [Crossref] [PubMed]
  11. D. Scoles, Y. N. Sulai, and A. Dubra, “In vivo dark-field imaging of the retinal pigment epithelium cell mosaic,” Biomed. Opt. Express 4, 1710–1723 (2013).
    [Crossref] [PubMed]
  12. A. Standard, “American national standard for the safe use of lasers.American National Standards Institute,” Inc., New York (1993).
  13. A. Dubra and Y. Sulai, “Reflective afocal broadband adaptive optics scanning ophthalmoscope,” Biomed. Opt. Express 2, 1757–1768 (2011).
    [Crossref] [PubMed]
  14. A. Dubra and Z. Harvey, “Registration of 2D images from fast scanning ophthalmic instruments,” in “Proc. of 2010 International Workshop on Biomedical Image Registration,” (Springer, 2010), pp. 60–71.
  15. D. G. Lowe, “Object recognition from local scale-invariant features,” in “Proc. of 1999 IEEE international Conference on Computer Vision,”, vol. 2 (Ieee, 1999), vol. 2, pp. 1150–1157.
  16. M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
    [Crossref]
  17. B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee, “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Med. Image Anal. 12, 26–41 (2008).
    [Crossref]
  18. A. Sotiras, C. Davatzikos, and N. Paragios, “Deformable medical image registration: A survey,” IEEE Trans. Med. Imag. 32, 1153–1190 (2013).
    [Crossref]
  19. A. Strehl and J. Ghosh, “Cluster ensembles—a knowledge reuse framework for combining multiple partitions,” J. Mach. Learn. Res. 3, 583–617 (2002).
  20. C. E. Shannon, “A mathematical theory of communication,” ACM SIGMOBILE Mobile Computing and Communications Review 5, 3–55 (2001).
    [Crossref]
  21. J. P. Pluim, J. A. Maintz, and M. A. Viergever, “Mutual-information-based registration of medical images: a survey,” IEEE Trans. Med. Imag. 22, 986–1004 (2003).
    [Crossref]
  22. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Und. 110, 346–359 (2008).
    [Crossref]
  23. J.-M. Morel and G. Yu, “Asift: A new framework for fully affine invariant image comparison,” SIAM J. Imaging Sci. 2, 438–469 (2009).
    [Crossref]
  24. J. C. Niebles, H. Wang, and L. Fei-Fei, “Unsupervised learning of human action categories using spatial-temporal words,” Int. J. Comput. Vision 79, 299–318 (2008).
    [Crossref]

2016 (1)

J. I. Morgan, “The fundus photo has met its match: optical coherence tomography and adaptive optics ophthalmoscopy are here to stay,” Ophthalmic. Physiol. Opt. 36, 218–239 (2016).
[Crossref] [PubMed]

2014 (1)

D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
[Crossref] [PubMed]

2013 (2)

D. Scoles, Y. N. Sulai, and A. Dubra, “In vivo dark-field imaging of the retinal pigment epithelium cell mosaic,” Biomed. Opt. Express 4, 1710–1723 (2013).
[Crossref] [PubMed]

A. Sotiras, C. Davatzikos, and N. Paragios, “Deformable medical image registration: A survey,” IEEE Trans. Med. Imag. 32, 1153–1190 (2013).
[Crossref]

2012 (2)

H. Li, J. Lu, G. Shi, and Y. Zhang, “Automatic montage of retinal images in adaptive optics confocal scanning laser ophthalmoscope,” Opt. Eng. 51, 057008 (2012).
[Crossref]

T. Y. Chui, D. A. VanNasdale, and S. A. Burns, “The use of forward scatter to improve retinal vascular imaging with an adaptive optics scanning laser ophthalmoscope,” Biomed. Opt. Express 3, 2537–2549 (2012).
[Crossref] [PubMed]

2011 (2)

2009 (2)

J. I. Morgan, A. Dubra, R. Wolfe, W. H. Merigan, and D. R. Williams, “In vivo autofluorescence imaging of the human and macaque retinal pigment epithelial cell mosaic,” Invest. Ophthalmol. Vis. Sci. 50, 1350–1359 (2009).
[Crossref]

J.-M. Morel and G. Yu, “Asift: A new framework for fully affine invariant image comparison,” SIAM J. Imaging Sci. 2, 438–469 (2009).
[Crossref]

2008 (3)

J. C. Niebles, H. Wang, and L. Fei-Fei, “Unsupervised learning of human action categories using spatial-temporal words,” Int. J. Comput. Vision 79, 299–318 (2008).
[Crossref]

B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee, “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Med. Image Anal. 12, 26–41 (2008).
[Crossref]

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Und. 110, 346–359 (2008).
[Crossref]

2003 (1)

J. P. Pluim, J. A. Maintz, and M. A. Viergever, “Mutual-information-based registration of medical images: a survey,” IEEE Trans. Med. Imag. 22, 986–1004 (2003).
[Crossref]

2002 (2)

A. Strehl and J. Ghosh, “Cluster ensembles—a knowledge reuse framework for combining multiple partitions,” J. Mach. Learn. Res. 3, 583–617 (2002).

A. Roorda, F. Romero-Borja, W. Donnelly, H. Queener, T. Hebert, and M. Campbell, “Adaptive optics scanning laser ophthalmoscopy,” Opt. Express 10, 405–412 (2002).
[Crossref] [PubMed]

2001 (2)

A. Roorda, A. B. Metha, P. Lennie, and D. R. Williams, “Packing arrangement of the three cone classes in primate retina,” Vision Res. 41, 1291–1306 (2001).
[Crossref] [PubMed]

C. E. Shannon, “A mathematical theory of communication,” ACM SIGMOBILE Mobile Computing and Communications Review 5, 3–55 (2001).
[Crossref]

1997 (1)

J. Liang, D. R. Williams, and D. T. Miller, “Supernormal vision and high-resolution retinal imaging through adaptive optics,” J. Opt. Soc. Am. 14, 2884–2892 (1997).
[Crossref]

1981 (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Avants, B. B.

B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee, “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Med. Image Anal. 12, 26–41 (2008).
[Crossref]

Bay, H.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Und. 110, 346–359 (2008).
[Crossref]

Bolles, R. C.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Burns, S. A.

Campbell, M.

Carroll, J.

D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
[Crossref] [PubMed]

A. Dubra, Y. Sulai, J. L. Norris, R. F. Cooper, A. M. Dubis, D. R. Williams, and J. Carroll, “Noninvasive imaging of the human rod photoreceptor mosaic using a confocal adaptive optics scanning ophthalmoscope,” Biomed. Opt. Express 2, 1864–1876 (2011).
[Crossref] [PubMed]

Chui, T. Y.

Cooper, R. F.

Curcio, C. A.

D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
[Crossref] [PubMed]

Davatzikos, C.

A. Sotiras, C. Davatzikos, and N. Paragios, “Deformable medical image registration: A survey,” IEEE Trans. Med. Imag. 32, 1153–1190 (2013).
[Crossref]

Donnelly, W.

Dubis, A. M.

Dubra, A.

D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
[Crossref] [PubMed]

D. Scoles, Y. N. Sulai, and A. Dubra, “In vivo dark-field imaging of the retinal pigment epithelium cell mosaic,” Biomed. Opt. Express 4, 1710–1723 (2013).
[Crossref] [PubMed]

A. Dubra, Y. Sulai, J. L. Norris, R. F. Cooper, A. M. Dubis, D. R. Williams, and J. Carroll, “Noninvasive imaging of the human rod photoreceptor mosaic using a confocal adaptive optics scanning ophthalmoscope,” Biomed. Opt. Express 2, 1864–1876 (2011).
[Crossref] [PubMed]

A. Dubra and Y. Sulai, “Reflective afocal broadband adaptive optics scanning ophthalmoscope,” Biomed. Opt. Express 2, 1757–1768 (2011).
[Crossref] [PubMed]

J. I. Morgan, A. Dubra, R. Wolfe, W. H. Merigan, and D. R. Williams, “In vivo autofluorescence imaging of the human and macaque retinal pigment epithelial cell mosaic,” Invest. Ophthalmol. Vis. Sci. 50, 1350–1359 (2009).
[Crossref]

A. Dubra and Z. Harvey, “Registration of 2D images from fast scanning ophthalmic instruments,” in “Proc. of 2010 International Workshop on Biomedical Image Registration,” (Springer, 2010), pp. 60–71.

Epstein, C. L.

B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee, “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Med. Image Anal. 12, 26–41 (2008).
[Crossref]

Ess, A.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Und. 110, 346–359 (2008).
[Crossref]

Fei-Fei, L.

J. C. Niebles, H. Wang, and L. Fei-Fei, “Unsupervised learning of human action categories using spatial-temporal words,” Int. J. Comput. Vision 79, 299–318 (2008).
[Crossref]

Fischler, M. A.

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Fishman, G. A.

D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
[Crossref] [PubMed]

Gee, J. C.

B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee, “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Med. Image Anal. 12, 26–41 (2008).
[Crossref]

Ghosh, J.

A. Strehl and J. Ghosh, “Cluster ensembles—a knowledge reuse framework for combining multiple partitions,” J. Mach. Learn. Res. 3, 583–617 (2002).

Grossman, M.

B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee, “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Med. Image Anal. 12, 26–41 (2008).
[Crossref]

Harvey, Z.

A. Dubra and Z. Harvey, “Registration of 2D images from fast scanning ophthalmic instruments,” in “Proc. of 2010 International Workshop on Biomedical Image Registration,” (Springer, 2010), pp. 60–71.

Hebert, T.

Ke, Y.

Y. Ke and R. Sukthankar, “PCA-SIFT: A more distinctive representation for local image descriptors,” in “Proc. of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition,”, vol. 2 (IEEE, 2004), vol. 2, pp. II–506.

Langlo, C. S.

D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
[Crossref] [PubMed]

Lennie, P.

A. Roorda, A. B. Metha, P. Lennie, and D. R. Williams, “Packing arrangement of the three cone classes in primate retina,” Vision Res. 41, 1291–1306 (2001).
[Crossref] [PubMed]

Li, H.

H. Li, J. Lu, G. Shi, and Y. Zhang, “Automatic montage of retinal images in adaptive optics confocal scanning laser ophthalmoscope,” Opt. Eng. 51, 057008 (2012).
[Crossref]

Liang, J.

J. Liang, D. R. Williams, and D. T. Miller, “Supernormal vision and high-resolution retinal imaging through adaptive optics,” J. Opt. Soc. Am. 14, 2884–2892 (1997).
[Crossref]

Lowe, D. G.

D. G. Lowe, “Object recognition from local scale-invariant features,” in “Proc. of 1999 IEEE international Conference on Computer Vision,”, vol. 2 (Ieee, 1999), vol. 2, pp. 1150–1157.

Lu, J.

H. Li, J. Lu, G. Shi, and Y. Zhang, “Automatic montage of retinal images in adaptive optics confocal scanning laser ophthalmoscope,” Opt. Eng. 51, 057008 (2012).
[Crossref]

Maintz, J. A.

J. P. Pluim, J. A. Maintz, and M. A. Viergever, “Mutual-information-based registration of medical images: a survey,” IEEE Trans. Med. Imag. 22, 986–1004 (2003).
[Crossref]

Merigan, W. H.

J. I. Morgan, A. Dubra, R. Wolfe, W. H. Merigan, and D. R. Williams, “In vivo autofluorescence imaging of the human and macaque retinal pigment epithelial cell mosaic,” Invest. Ophthalmol. Vis. Sci. 50, 1350–1359 (2009).
[Crossref]

Metha, A. B.

A. Roorda, A. B. Metha, P. Lennie, and D. R. Williams, “Packing arrangement of the three cone classes in primate retina,” Vision Res. 41, 1291–1306 (2001).
[Crossref] [PubMed]

Miller, D. T.

J. Liang, D. R. Williams, and D. T. Miller, “Supernormal vision and high-resolution retinal imaging through adaptive optics,” J. Opt. Soc. Am. 14, 2884–2892 (1997).
[Crossref]

Morel, J.-M.

J.-M. Morel and G. Yu, “Asift: A new framework for fully affine invariant image comparison,” SIAM J. Imaging Sci. 2, 438–469 (2009).
[Crossref]

Morgan, J. I.

J. I. Morgan, “The fundus photo has met its match: optical coherence tomography and adaptive optics ophthalmoscopy are here to stay,” Ophthalmic. Physiol. Opt. 36, 218–239 (2016).
[Crossref] [PubMed]

J. I. Morgan, A. Dubra, R. Wolfe, W. H. Merigan, and D. R. Williams, “In vivo autofluorescence imaging of the human and macaque retinal pigment epithelial cell mosaic,” Invest. Ophthalmol. Vis. Sci. 50, 1350–1359 (2009).
[Crossref]

Niebles, J. C.

J. C. Niebles, H. Wang, and L. Fei-Fei, “Unsupervised learning of human action categories using spatial-temporal words,” Int. J. Comput. Vision 79, 299–318 (2008).
[Crossref]

Norris, J. L.

Paragios, N.

A. Sotiras, C. Davatzikos, and N. Paragios, “Deformable medical image registration: A survey,” IEEE Trans. Med. Imag. 32, 1153–1190 (2013).
[Crossref]

Pluim, J. P.

J. P. Pluim, J. A. Maintz, and M. A. Viergever, “Mutual-information-based registration of medical images: a survey,” IEEE Trans. Med. Imag. 22, 986–1004 (2003).
[Crossref]

Queener, H.

Romero-Borja, F.

Roorda, A.

A. Roorda, F. Romero-Borja, W. Donnelly, H. Queener, T. Hebert, and M. Campbell, “Adaptive optics scanning laser ophthalmoscopy,” Opt. Express 10, 405–412 (2002).
[Crossref] [PubMed]

A. Roorda, A. B. Metha, P. Lennie, and D. R. Williams, “Packing arrangement of the three cone classes in primate retina,” Vision Res. 41, 1291–1306 (2001).
[Crossref] [PubMed]

Scoles, D.

D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
[Crossref] [PubMed]

D. Scoles, Y. N. Sulai, and A. Dubra, “In vivo dark-field imaging of the retinal pigment epithelium cell mosaic,” Biomed. Opt. Express 4, 1710–1723 (2013).
[Crossref] [PubMed]

Shannon, C. E.

C. E. Shannon, “A mathematical theory of communication,” ACM SIGMOBILE Mobile Computing and Communications Review 5, 3–55 (2001).
[Crossref]

Shi, G.

H. Li, J. Lu, G. Shi, and Y. Zhang, “Automatic montage of retinal images in adaptive optics confocal scanning laser ophthalmoscope,” Opt. Eng. 51, 057008 (2012).
[Crossref]

Sotiras, A.

A. Sotiras, C. Davatzikos, and N. Paragios, “Deformable medical image registration: A survey,” IEEE Trans. Med. Imag. 32, 1153–1190 (2013).
[Crossref]

Strehl, A.

A. Strehl and J. Ghosh, “Cluster ensembles—a knowledge reuse framework for combining multiple partitions,” J. Mach. Learn. Res. 3, 583–617 (2002).

Sukthankar, R.

Y. Ke and R. Sukthankar, “PCA-SIFT: A more distinctive representation for local image descriptors,” in “Proc. of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition,”, vol. 2 (IEEE, 2004), vol. 2, pp. II–506.

Sulai, Y.

Sulai, Y. N.

D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
[Crossref] [PubMed]

D. Scoles, Y. N. Sulai, and A. Dubra, “In vivo dark-field imaging of the retinal pigment epithelium cell mosaic,” Biomed. Opt. Express 4, 1710–1723 (2013).
[Crossref] [PubMed]

Tuytelaars, T.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Und. 110, 346–359 (2008).
[Crossref]

Van Gool, L.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Und. 110, 346–359 (2008).
[Crossref]

VanNasdale, D. A.

Viergever, M. A.

J. P. Pluim, J. A. Maintz, and M. A. Viergever, “Mutual-information-based registration of medical images: a survey,” IEEE Trans. Med. Imag. 22, 986–1004 (2003).
[Crossref]

Wang, H.

J. C. Niebles, H. Wang, and L. Fei-Fei, “Unsupervised learning of human action categories using spatial-temporal words,” Int. J. Comput. Vision 79, 299–318 (2008).
[Crossref]

Williams, D. R.

A. Dubra, Y. Sulai, J. L. Norris, R. F. Cooper, A. M. Dubis, D. R. Williams, and J. Carroll, “Noninvasive imaging of the human rod photoreceptor mosaic using a confocal adaptive optics scanning ophthalmoscope,” Biomed. Opt. Express 2, 1864–1876 (2011).
[Crossref] [PubMed]

J. I. Morgan, A. Dubra, R. Wolfe, W. H. Merigan, and D. R. Williams, “In vivo autofluorescence imaging of the human and macaque retinal pigment epithelial cell mosaic,” Invest. Ophthalmol. Vis. Sci. 50, 1350–1359 (2009).
[Crossref]

A. Roorda, A. B. Metha, P. Lennie, and D. R. Williams, “Packing arrangement of the three cone classes in primate retina,” Vision Res. 41, 1291–1306 (2001).
[Crossref] [PubMed]

J. Liang, D. R. Williams, and D. T. Miller, “Supernormal vision and high-resolution retinal imaging through adaptive optics,” J. Opt. Soc. Am. 14, 2884–2892 (1997).
[Crossref]

Wolfe, R.

J. I. Morgan, A. Dubra, R. Wolfe, W. H. Merigan, and D. R. Williams, “In vivo autofluorescence imaging of the human and macaque retinal pigment epithelial cell mosaic,” Invest. Ophthalmol. Vis. Sci. 50, 1350–1359 (2009).
[Crossref]

Yu, G.

J.-M. Morel and G. Yu, “Asift: A new framework for fully affine invariant image comparison,” SIAM J. Imaging Sci. 2, 438–469 (2009).
[Crossref]

Zhang, Y.

H. Li, J. Lu, G. Shi, and Y. Zhang, “Automatic montage of retinal images in adaptive optics confocal scanning laser ophthalmoscope,” Opt. Eng. 51, 057008 (2012).
[Crossref]

ACM SIGMOBILE Mobile Computing and Communications Review (1)

C. E. Shannon, “A mathematical theory of communication,” ACM SIGMOBILE Mobile Computing and Communications Review 5, 3–55 (2001).
[Crossref]

Biomed. Opt. Express (4)

Commun. ACM (1)

M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981).
[Crossref]

Comput. Vis. Image Und. (1)

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Und. 110, 346–359 (2008).
[Crossref]

IEEE Trans. Med. Imag. (2)

J. P. Pluim, J. A. Maintz, and M. A. Viergever, “Mutual-information-based registration of medical images: a survey,” IEEE Trans. Med. Imag. 22, 986–1004 (2003).
[Crossref]

A. Sotiras, C. Davatzikos, and N. Paragios, “Deformable medical image registration: A survey,” IEEE Trans. Med. Imag. 32, 1153–1190 (2013).
[Crossref]

Int. J. Comput. Vision (1)

J. C. Niebles, H. Wang, and L. Fei-Fei, “Unsupervised learning of human action categories using spatial-temporal words,” Int. J. Comput. Vision 79, 299–318 (2008).
[Crossref]

Invest. Ophthalmol. Vis. Sci. (2)

D. Scoles, Y. N. Sulai, C. S. Langlo, G. A. Fishman, C. A. Curcio, J. Carroll, and A. Dubra, “In vivo imaging of human cone photoreceptor inner segments,” Invest. Ophthalmol. Vis. Sci. 55, 4244–4251 (2014).
[Crossref] [PubMed]

J. I. Morgan, A. Dubra, R. Wolfe, W. H. Merigan, and D. R. Williams, “In vivo autofluorescence imaging of the human and macaque retinal pigment epithelial cell mosaic,” Invest. Ophthalmol. Vis. Sci. 50, 1350–1359 (2009).
[Crossref]

J. Mach. Learn. Res. (1)

A. Strehl and J. Ghosh, “Cluster ensembles—a knowledge reuse framework for combining multiple partitions,” J. Mach. Learn. Res. 3, 583–617 (2002).

J. Opt. Soc. Am. (1)

J. Liang, D. R. Williams, and D. T. Miller, “Supernormal vision and high-resolution retinal imaging through adaptive optics,” J. Opt. Soc. Am. 14, 2884–2892 (1997).
[Crossref]

Med. Image Anal. (1)

B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee, “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Med. Image Anal. 12, 26–41 (2008).
[Crossref]

Ophthalmic. Physiol. Opt. (1)

J. I. Morgan, “The fundus photo has met its match: optical coherence tomography and adaptive optics ophthalmoscopy are here to stay,” Ophthalmic. Physiol. Opt. 36, 218–239 (2016).
[Crossref] [PubMed]

Opt. Eng. (1)

H. Li, J. Lu, G. Shi, and Y. Zhang, “Automatic montage of retinal images in adaptive optics confocal scanning laser ophthalmoscope,” Opt. Eng. 51, 057008 (2012).
[Crossref]

Opt. Express (1)

SIAM J. Imaging Sci. (1)

J.-M. Morel and G. Yu, “Asift: A new framework for fully affine invariant image comparison,” SIAM J. Imaging Sci. 2, 438–469 (2009).
[Crossref]

Vision Res. (1)

A. Roorda, A. B. Metha, P. Lennie, and D. R. Williams, “Packing arrangement of the three cone classes in primate retina,” Vision Res. 41, 1291–1306 (2001).
[Crossref] [PubMed]

Other (4)

Y. Ke and R. Sukthankar, “PCA-SIFT: A more distinctive representation for local image descriptors,” in “Proc. of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition,”, vol. 2 (IEEE, 2004), vol. 2, pp. II–506.

A. Dubra and Z. Harvey, “Registration of 2D images from fast scanning ophthalmic instruments,” in “Proc. of 2010 International Workshop on Biomedical Image Registration,” (Springer, 2010), pp. 60–71.

D. G. Lowe, “Object recognition from local scale-invariant features,” in “Proc. of 1999 IEEE international Conference on Computer Vision,”, vol. 2 (Ieee, 1999), vol. 2, pp. 1150–1157.

A. Standard, “American national standard for the safe use of lasers.American National Standards Institute,” Inc., New York (1993).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1 Temporal arm of a set of confocal AO images of cone outer segments (a) before and (b) after manual montaging. In (a), individual image thumbnails are shown in order of their nominal retinal locations, as determined by the location of the fixation point that the subject was instructed to look at during image acquisition. The larger images show more detail for two adjacent images. In (b), a manual montage of this image set is shown.
Fig. 2
Fig. 2 Examples of three simultaneously acquired AO image modalities (confocal, split detection, and dark field), at two adjacent nominal locations. The confocal images are the same as shown in detail in Fig. 1.
Fig. 3
Fig. 3 Shown are feature matches found by SIFT on the confocal, split detection, and dark field images between two adjacent nominal locations. The individual lines connect matched features in the two images. Note that most matches suggest a consistent global coordinate transformation between the two images, but that in each case there are also outlier matches.
Fig. 4
Fig. 4 Shown are (a) the remaining SIFT matches from Fig. 3 after using multi-modal RANSAC to remove outliers from all three modalities simultaneously, and (b) the montage created by applying the transformation learned from the matching features to the second image. For visualization, the intensities in the overlapping regions were averaged at each pixel to show the alignment of the corresponding structures in the two images.
Fig. 5
Fig. 5 An example of a full manual and automated (rigid) montage of the same dataset.
Fig. 6
Fig. 6 Average normalized cross-correlation of each overlapping region in the manual and automated (rigid) montage shown in Fig. 5
Fig. 7
Fig. 7 Plots showing the pairwise alignment accuracy using different numbers of columns (pixels) of overlap between the images. Shown are the results when aligning images from the (a) control, (b) CSCR, and (c) RP datasets when using each modality [confocal, split detection (SD), dark field (DF)] individually or all three modalities (All Modalities) together as inputs in the algorithm.
Fig. 8
Fig. 8 Shown are (a) two adjacent confocal foveal images, (b) the SIFT matches found between the images, and (c) the remaining SIFT features after RANSAC.

Tables (3)

Tables Icon

Table 1 Average (standard deviation) normalized cross-correlation (NCC) and normalized mutual information (NMI) of all overlapping regions in the confocal modality over the 11 datasets after manual and automated montaging. *Significant (α = .05) improvements when compared to manual results using paired Wilcoxon test.

Tables Icon

Table 2 Average (and standard deviation) normalized cross-correlation (NCC) and normalized mutual information (NMI) of all overlapping regions over the 11 datasets when using the automated algorithm with individual or multiple modalities as inputs. *Significant (α = .05) improvements when compared to single modality results using paired Wilcoxon test.

Tables Icon

Table 3 Counts of the correct and incorrect discontinuities found in the manual and automated montages, over all datasets.

Equations (19)

Equations on this page are rendered with MathJax. Learn more.

x i = T ref i ( x ref ) ,
I i , m ref ( x ref ) = I i , m ( T ref i ( x ref ) )
I i , m ( f i , n ) I ref , m ( f ref , n ) ,
F i = T ref i F ref ,
F i = [ x i , 1 y i , 1 1 x i , 2 y i , 2 1 x i , n y i , n 1 ]
F ref = [ x ref , 1 y ref , 1 1 x ref , 2 y ref , 2 1 x ref , N y ref , N 1 ]
T Trans = [ 1 0 a 0 1 b 0 0 1 ] ,
T Rigid = [ cos θ sin θ a sin θ cos θ b 0 0 1 ] ,
T ref ( i + 1 ) ( x ref ) = T i ( i + 1 ) ( T ref i ( x ref ) ) .
| f i , n T R ( f ref , n ) | 2
S R = n = 1 N 1 ( < σ 2 ) ( | f i , n T R ( f ref , n ) | 2 ) ,
f i , m , n = ( x i , m , n , y i , m , n )
f ref , m , n = ( x ref , m , n , y ref , m , n ) .
S R , M = m = 1 M n = 1 N 1 ( < σ 2 ) ( | f i , m , n T R ( f ref , m , n ) | 2 ) ,
NCC = 1 n x [ A ( x ) a ¯ ] [ B ( x ) b ¯ ] σ A σ B ,
NMI = H ( A ) + H ( B ) H ( A , B ) H ( A ) H ( B ) ,
H ( A ) = i = 1 W P A ( a i ) log P A ( a i ) ,
H ( B ) = j = 1 V P B ( b j ) log P B ( b j ) ,
H ( A , B ) = i = 1 W j = 1 V P A , B ( a i , b j ) log P A , B ( a i , b j )

Metrics