Abstract

Microendoscopes allow clinicians to view subcellular features in vivo and in real-time, but their field-of-view is inherently limited by the small size of the probe’s distal end. Video mosaicing has emerged as an effective technique to increase the acquired image size. Current implementations are performed post-procedure, which removes the benefits of live imaging. In this manuscript we present an algorithm for real-time video mosaicing using a low-cost high-resolution microendoscope. We present algorithm execution times and show image results obtained from in vivo tissue.

© 2012 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods2(12), 941–950 (2005).
    [CrossRef] [PubMed]
  2. K. E. Loewke, D. B. Camarillo, C. A. Jobst, and J. K. Salisbury, “Real-time image mosaicing for medical applications,” Stud. Health Technol. Inform.125, 304–309 (2007).
    [PubMed]
  3. M. L. Hearp, A. M. Locante, M. Ben-Rubin, R. Dietrich, and O. David, “Validity of sampling error as a cause of noncorrelation,” Cancer111(5), 275–279 (2007).
    [CrossRef] [PubMed]
  4. V. Becker, T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining, “High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video),” Gastrointest. Endosc.66(5), 1001–1007 (2007).
    [CrossRef] [PubMed]
  5. T. Vercauteren, A. Perchant, G. Malandain, X. Pennec, and N. Ayache, “Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy,” Med. Image Anal.10(5), 673–692 (2006).
    [CrossRef] [PubMed]
  6. K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
    [CrossRef] [PubMed]
  7. T. Vercauteren, A. Meining, F. Lacombe, and A. Perchant, “Real time autonomous video image registration for endomicroscopy: fighting the compromises,” in Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XV (International Society for Optical Engineering, San Jose, CA, USA, 2008).
  8. K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
    [CrossRef] [PubMed]
  9. A. Behrens, M. Bommes, T. Stehle, S. Gross, S. Leonhardt, and T. Aach, “Real-time image composition of bladder mosaics in fluorescence endoscopy,” Comput. Sci. Res. Dev.26, 51–64 (2011).
  10. T. J. Muldoon, M. C. Pierce, D. L. Nida, M. D. Williams, A. Gillenwater, and R. Richards-Kortum, “Subcellular-resolution molecular imaging within living tissue by fiber microendoscopy,” Opt. Express15(25), 16413–16423 (2007).
    [CrossRef] [PubMed]
  11. T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. Stepanek, R. Richards-Kortum, and A. M. Gillenwater, “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck34(3), 305–312 (2012).
    [CrossRef] [PubMed]
  12. M. Pierce, D. Yu, and R. Richards-Kortum, “High-resolution fiber-optic microendoscopy for in situ cellular imaging,” J. Vis. Exp.47(47), 2306 (2011).
    [PubMed]
  13. M. Elter, S. Rupp, and C. Winter, “Physically motivated reconstruction of fiberscopic images,” in 18th International Conference on Pattern Recognition (Hong Kong, 2006), pp. 599–602.
  14. B. L. Luck, K. D. Carlson, A. C. Bovik, and R. R. Richards-Kortum, “An image model and segmentation algorithm for reflectance confocal images of in vivo cervical tissue,” IEEE Trans. Image Process.14(9), 1265–1276 (2005).
    [CrossRef] [PubMed]
  15. S. Rupp, C. Winter, and M. Elter, “Evaluation of spatial interpolation strategies for the removal of comb-structure in fiber-optic images ” in 31st Annual International Conference of the IEEE EMBS (Minneapolis, Minnesota, USA, 2009), pp. 3677–3680.
  16. P. Soille, Morphological Image Analysis: Principles and Applications (Springer, 1999).
  17. M. Guizar-Sicairos, S. T. Thurman, and J. R. Fienup, “Efficient subpixel image registration algorithms,” Opt. Lett.33(2), 156–158 (2008).
    [CrossRef] [PubMed]

2012 (1)

T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. Stepanek, R. Richards-Kortum, and A. M. Gillenwater, “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck34(3), 305–312 (2012).
[CrossRef] [PubMed]

2011 (4)

M. Pierce, D. Yu, and R. Richards-Kortum, “High-resolution fiber-optic microendoscopy for in situ cellular imaging,” J. Vis. Exp.47(47), 2306 (2011).
[PubMed]

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
[CrossRef] [PubMed]

A. Behrens, M. Bommes, T. Stehle, S. Gross, S. Leonhardt, and T. Aach, “Real-time image composition of bladder mosaics in fluorescence endoscopy,” Comput. Sci. Res. Dev.26, 51–64 (2011).

2008 (1)

2007 (4)

T. J. Muldoon, M. C. Pierce, D. L. Nida, M. D. Williams, A. Gillenwater, and R. Richards-Kortum, “Subcellular-resolution molecular imaging within living tissue by fiber microendoscopy,” Opt. Express15(25), 16413–16423 (2007).
[CrossRef] [PubMed]

K. E. Loewke, D. B. Camarillo, C. A. Jobst, and J. K. Salisbury, “Real-time image mosaicing for medical applications,” Stud. Health Technol. Inform.125, 304–309 (2007).
[PubMed]

M. L. Hearp, A. M. Locante, M. Ben-Rubin, R. Dietrich, and O. David, “Validity of sampling error as a cause of noncorrelation,” Cancer111(5), 275–279 (2007).
[CrossRef] [PubMed]

V. Becker, T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining, “High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video),” Gastrointest. Endosc.66(5), 1001–1007 (2007).
[CrossRef] [PubMed]

2006 (1)

T. Vercauteren, A. Perchant, G. Malandain, X. Pennec, and N. Ayache, “Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy,” Med. Image Anal.10(5), 673–692 (2006).
[CrossRef] [PubMed]

2005 (2)

B. L. Luck, K. D. Carlson, A. C. Bovik, and R. R. Richards-Kortum, “An image model and segmentation algorithm for reflectance confocal images of in vivo cervical tissue,” IEEE Trans. Image Process.14(9), 1265–1276 (2005).
[CrossRef] [PubMed]

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods2(12), 941–950 (2005).
[CrossRef] [PubMed]

Aach, T.

A. Behrens, M. Bommes, T. Stehle, S. Gross, S. Leonhardt, and T. Aach, “Real-time image composition of bladder mosaics in fluorescence endoscopy,” Comput. Sci. Res. Dev.26, 51–64 (2011).

Adams, W.

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Ayache, N.

T. Vercauteren, A. Perchant, G. Malandain, X. Pennec, and N. Ayache, “Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy,” Med. Image Anal.10(5), 673–692 (2006).
[CrossRef] [PubMed]

Beck, A. H.

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Becker, V.

V. Becker, T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining, “High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video),” Gastrointest. Endosc.66(5), 1001–1007 (2007).
[CrossRef] [PubMed]

Behrens, A.

A. Behrens, M. Bommes, T. Stehle, S. Gross, S. Leonhardt, and T. Aach, “Real-time image composition of bladder mosaics in fluorescence endoscopy,” Comput. Sci. Res. Dev.26, 51–64 (2011).

Ben-Rubin, M.

M. L. Hearp, A. M. Locante, M. Ben-Rubin, R. Dietrich, and O. David, “Validity of sampling error as a cause of noncorrelation,” Cancer111(5), 275–279 (2007).
[CrossRef] [PubMed]

Bommes, M.

A. Behrens, M. Bommes, T. Stehle, S. Gross, S. Leonhardt, and T. Aach, “Real-time image composition of bladder mosaics in fluorescence endoscopy,” Comput. Sci. Res. Dev.26, 51–64 (2011).

Bovik, A. C.

B. L. Luck, K. D. Carlson, A. C. Bovik, and R. R. Richards-Kortum, “An image model and segmentation algorithm for reflectance confocal images of in vivo cervical tissue,” IEEE Trans. Image Process.14(9), 1265–1276 (2005).
[CrossRef] [PubMed]

Camarillo, D. B.

K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
[CrossRef] [PubMed]

K. E. Loewke, D. B. Camarillo, C. A. Jobst, and J. K. Salisbury, “Real-time image mosaicing for medical applications,” Stud. Health Technol. Inform.125, 304–309 (2007).
[PubMed]

Carlson, K. D.

B. L. Luck, K. D. Carlson, A. C. Bovik, and R. R. Richards-Kortum, “An image model and segmentation algorithm for reflectance confocal images of in vivo cervical tissue,” IEEE Trans. Image Process.14(9), 1265–1276 (2005).
[CrossRef] [PubMed]

Cheung, E. L.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods2(12), 941–950 (2005).
[CrossRef] [PubMed]

Cocker, E. D.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods2(12), 941–950 (2005).
[CrossRef] [PubMed]

Contag, C. H.

K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
[CrossRef] [PubMed]

David, O.

M. L. Hearp, A. M. Locante, M. Ben-Rubin, R. Dietrich, and O. David, “Validity of sampling error as a cause of noncorrelation,” Cancer111(5), 275–279 (2007).
[CrossRef] [PubMed]

Dietrich, R.

M. L. Hearp, A. M. Locante, M. Ben-Rubin, R. Dietrich, and O. David, “Validity of sampling error as a cause of noncorrelation,” Cancer111(5), 275–279 (2007).
[CrossRef] [PubMed]

Fienup, J. R.

Flusberg, B. A.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods2(12), 941–950 (2005).
[CrossRef] [PubMed]

Gillenwater, A.

Gillenwater, A. M.

T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. Stepanek, R. Richards-Kortum, and A. M. Gillenwater, “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck34(3), 305–312 (2012).
[CrossRef] [PubMed]

Gross, S.

A. Behrens, M. Bommes, T. Stehle, S. Gross, S. Leonhardt, and T. Aach, “Real-time image composition of bladder mosaics in fluorescence endoscopy,” Comput. Sci. Res. Dev.26, 51–64 (2011).

Guizar-Sicairos, M.

Hearp, M. L.

M. L. Hearp, A. M. Locante, M. Ben-Rubin, R. Dietrich, and O. David, “Validity of sampling error as a cause of noncorrelation,” Cancer111(5), 275–279 (2007).
[CrossRef] [PubMed]

Jensen, K. C.

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Jobst, C. A.

K. E. Loewke, D. B. Camarillo, C. A. Jobst, and J. K. Salisbury, “Real-time image mosaicing for medical applications,” Stud. Health Technol. Inform.125, 304–309 (2007).
[PubMed]

Jung, J. C.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods2(12), 941–950 (2005).
[CrossRef] [PubMed]

Leonhardt, S.

A. Behrens, M. Bommes, T. Stehle, S. Gross, S. Leonhardt, and T. Aach, “Real-time image composition of bladder mosaics in fluorescence endoscopy,” Comput. Sci. Res. Dev.26, 51–64 (2011).

Liao, J. C.

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Liu, J. J.

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Locante, A. M.

M. L. Hearp, A. M. Locante, M. Ben-Rubin, R. Dietrich, and O. David, “Validity of sampling error as a cause of noncorrelation,” Cancer111(5), 275–279 (2007).
[CrossRef] [PubMed]

Loewke, K. E.

K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
[CrossRef] [PubMed]

K. E. Loewke, D. B. Camarillo, C. A. Jobst, and J. K. Salisbury, “Real-time image mosaicing for medical applications,” Stud. Health Technol. Inform.125, 304–309 (2007).
[PubMed]

Luck, B. L.

B. L. Luck, K. D. Carlson, A. C. Bovik, and R. R. Richards-Kortum, “An image model and segmentation algorithm for reflectance confocal images of in vivo cervical tissue,” IEEE Trans. Image Process.14(9), 1265–1276 (2005).
[CrossRef] [PubMed]

Mach, K. E.

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Malandain, G.

T. Vercauteren, A. Perchant, G. Malandain, X. Pennec, and N. Ayache, “Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy,” Med. Image Anal.10(5), 673–692 (2006).
[CrossRef] [PubMed]

Mandella, M. J.

K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
[CrossRef] [PubMed]

Meining, A.

V. Becker, T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining, “High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video),” Gastrointest. Endosc.66(5), 1001–1007 (2007).
[CrossRef] [PubMed]

Muldoon, T. J.

T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. Stepanek, R. Richards-Kortum, and A. M. Gillenwater, “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck34(3), 305–312 (2012).
[CrossRef] [PubMed]

T. J. Muldoon, M. C. Pierce, D. L. Nida, M. D. Williams, A. Gillenwater, and R. Richards-Kortum, “Subcellular-resolution molecular imaging within living tissue by fiber microendoscopy,” Opt. Express15(25), 16413–16423 (2007).
[CrossRef] [PubMed]

Nida, D. L.

Pan, Y.

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Pennec, X.

T. Vercauteren, A. Perchant, G. Malandain, X. Pennec, and N. Ayache, “Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy,” Med. Image Anal.10(5), 673–692 (2006).
[CrossRef] [PubMed]

Perchant, A.

T. Vercauteren, A. Perchant, G. Malandain, X. Pennec, and N. Ayache, “Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy,” Med. Image Anal.10(5), 673–692 (2006).
[CrossRef] [PubMed]

Pierce, M.

M. Pierce, D. Yu, and R. Richards-Kortum, “High-resolution fiber-optic microendoscopy for in situ cellular imaging,” J. Vis. Exp.47(47), 2306 (2011).
[PubMed]

Pierce, M. C.

Piyawattanametha, W.

K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
[CrossRef] [PubMed]

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods2(12), 941–950 (2005).
[CrossRef] [PubMed]

Prinz, C.

V. Becker, T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining, “High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video),” Gastrointest. Endosc.66(5), 1001–1007 (2007).
[CrossRef] [PubMed]

Richards-Kortum, R.

T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. Stepanek, R. Richards-Kortum, and A. M. Gillenwater, “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck34(3), 305–312 (2012).
[CrossRef] [PubMed]

M. Pierce, D. Yu, and R. Richards-Kortum, “High-resolution fiber-optic microendoscopy for in situ cellular imaging,” J. Vis. Exp.47(47), 2306 (2011).
[PubMed]

T. J. Muldoon, M. C. Pierce, D. L. Nida, M. D. Williams, A. Gillenwater, and R. Richards-Kortum, “Subcellular-resolution molecular imaging within living tissue by fiber microendoscopy,” Opt. Express15(25), 16413–16423 (2007).
[CrossRef] [PubMed]

Richards-Kortum, R. R.

B. L. Luck, K. D. Carlson, A. C. Bovik, and R. R. Richards-Kortum, “An image model and segmentation algorithm for reflectance confocal images of in vivo cervical tissue,” IEEE Trans. Image Process.14(9), 1265–1276 (2005).
[CrossRef] [PubMed]

Roblyer, D.

T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. Stepanek, R. Richards-Kortum, and A. M. Gillenwater, “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck34(3), 305–312 (2012).
[CrossRef] [PubMed]

Salisbury, J. K.

K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
[CrossRef] [PubMed]

K. E. Loewke, D. B. Camarillo, C. A. Jobst, and J. K. Salisbury, “Real-time image mosaicing for medical applications,” Stud. Health Technol. Inform.125, 304–309 (2007).
[PubMed]

Schmid, R. M.

V. Becker, T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining, “High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video),” Gastrointest. Endosc.66(5), 1001–1007 (2007).
[CrossRef] [PubMed]

Schnitzer, M. J.

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods2(12), 941–950 (2005).
[CrossRef] [PubMed]

Sonn, G. A.

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Stehle, T.

A. Behrens, M. Bommes, T. Stehle, S. Gross, S. Leonhardt, and T. Aach, “Real-time image composition of bladder mosaics in fluorescence endoscopy,” Comput. Sci. Res. Dev.26, 51–64 (2011).

Stepanek, V. M.

T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. Stepanek, R. Richards-Kortum, and A. M. Gillenwater, “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck34(3), 305–312 (2012).
[CrossRef] [PubMed]

Thrun, S.

K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
[CrossRef] [PubMed]

Thurman, S. T.

Vercauteren, T.

V. Becker, T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining, “High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video),” Gastrointest. Endosc.66(5), 1001–1007 (2007).
[CrossRef] [PubMed]

T. Vercauteren, A. Perchant, G. Malandain, X. Pennec, and N. Ayache, “Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy,” Med. Image Anal.10(5), 673–692 (2006).
[CrossRef] [PubMed]

von Weyhern, C. H.

V. Becker, T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining, “High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video),” Gastrointest. Endosc.66(5), 1001–1007 (2007).
[CrossRef] [PubMed]

Williams, M. D.

T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. Stepanek, R. Richards-Kortum, and A. M. Gillenwater, “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck34(3), 305–312 (2012).
[CrossRef] [PubMed]

T. J. Muldoon, M. C. Pierce, D. L. Nida, M. D. Williams, A. Gillenwater, and R. Richards-Kortum, “Subcellular-resolution molecular imaging within living tissue by fiber microendoscopy,” Opt. Express15(25), 16413–16423 (2007).
[CrossRef] [PubMed]

Wu, K.

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Yu, D.

M. Pierce, D. Yu, and R. Richards-Kortum, “High-resolution fiber-optic microendoscopy for in situ cellular imaging,” J. Vis. Exp.47(47), 2306 (2011).
[PubMed]

Cancer (1)

M. L. Hearp, A. M. Locante, M. Ben-Rubin, R. Dietrich, and O. David, “Validity of sampling error as a cause of noncorrelation,” Cancer111(5), 275–279 (2007).
[CrossRef] [PubMed]

Comput. Sci. Res. Dev. (1)

A. Behrens, M. Bommes, T. Stehle, S. Gross, S. Leonhardt, and T. Aach, “Real-time image composition of bladder mosaics in fluorescence endoscopy,” Comput. Sci. Res. Dev.26, 51–64 (2011).

Gastrointest. Endosc. (1)

V. Becker, T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining, “High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video),” Gastrointest. Endosc.66(5), 1001–1007 (2007).
[CrossRef] [PubMed]

Head Neck (1)

T. J. Muldoon, D. Roblyer, M. D. Williams, V. M. Stepanek, R. Richards-Kortum, and A. M. Gillenwater, “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck34(3), 305–312 (2012).
[CrossRef] [PubMed]

IEEE Trans. Biomed. Eng. (1)

K. E. Loewke, D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury, “In vivo micro-image mosaicing,” IEEE Trans. Biomed. Eng.58(1), 159–171 (2011).
[CrossRef] [PubMed]

IEEE Trans. Image Process. (1)

B. L. Luck, K. D. Carlson, A. C. Bovik, and R. R. Richards-Kortum, “An image model and segmentation algorithm for reflectance confocal images of in vivo cervical tissue,” IEEE Trans. Image Process.14(9), 1265–1276 (2005).
[CrossRef] [PubMed]

J. Vis. Exp. (1)

M. Pierce, D. Yu, and R. Richards-Kortum, “High-resolution fiber-optic microendoscopy for in situ cellular imaging,” J. Vis. Exp.47(47), 2306 (2011).
[PubMed]

Med. Image Anal. (1)

T. Vercauteren, A. Perchant, G. Malandain, X. Pennec, and N. Ayache, “Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy,” Med. Image Anal.10(5), 673–692 (2006).
[CrossRef] [PubMed]

Nat. Methods (1)

B. A. Flusberg, E. D. Cocker, W. Piyawattanametha, J. C. Jung, E. L. Cheung, and M. J. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods2(12), 941–950 (2005).
[CrossRef] [PubMed]

Opt. Express (1)

Opt. Lett. (1)

Stud. Health Technol. Inform. (1)

K. E. Loewke, D. B. Camarillo, C. A. Jobst, and J. K. Salisbury, “Real-time image mosaicing for medical applications,” Stud. Health Technol. Inform.125, 304–309 (2007).
[PubMed]

Urology (1)

K. Wu, J. J. Liu, W. Adams, G. A. Sonn, K. E. Mach, Y. Pan, A. H. Beck, K. C. Jensen, and J. C. Liao, “Dynamic real-time microscopy of the urinary tract using confocal laser endomicroscopy,” Urology78(1), 225–231 (2011).
[CrossRef] [PubMed]

Other (4)

T. Vercauteren, A. Meining, F. Lacombe, and A. Perchant, “Real time autonomous video image registration for endomicroscopy: fighting the compromises,” in Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XV (International Society for Optical Engineering, San Jose, CA, USA, 2008).

M. Elter, S. Rupp, and C. Winter, “Physically motivated reconstruction of fiberscopic images,” in 18th International Conference on Pattern Recognition (Hong Kong, 2006), pp. 599–602.

S. Rupp, C. Winter, and M. Elter, “Evaluation of spatial interpolation strategies for the removal of comb-structure in fiber-optic images ” in 31st Annual International Conference of the IEEE EMBS (Minneapolis, Minnesota, USA, 2009), pp. 3677–3680.

P. Soille, Morphological Image Analysis: Principles and Applications (Springer, 1999).

Supplementary Material (4)

» Media 1: AVI (697 KB)     
» Media 2: AVI (1159 KB)     
» Media 3: AVI (1214 KB)     
» Media 4: AVI (688 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1
Fig. 1

The high-resolution microendoscope (HRME) is a portable fluorescence imaging device that provides in vivo subcellular detail with a field-of-view of <1 mm. (B) Real-time video mosaicing increases the field-of-view (Media 1).

Fig. 2
Fig. 2

Fiber pattern removal. Left, the raw HRME image shows squamous epithelial cells in the oral cavity. Nuclei are labeled with the 0.01% Proflavine. The intensity information from the sample is “pixelated” by the 30,000-element fiber bundle. Right, the fiber pixelation pattern is removed using two-dimensional interpolation. First, the fiber center locations are identified using a regional maxima function. A Voronoi tessellation is then constructed from these points in order to determine the nearest-neighbor fiber pixel for every non-fiber pixel. A lookup table is determined from this tessellation and is used to reconstruct fiber-less HRME images in real-time. Other interpolation methods, such as bilinear interpolation, can be used but have longer computation times.

Fig. 3
Fig. 3

The HRME was translated over fluorescently-labeled lens paper to build a mosaic. (A) When video frames are inserted into the mosaic using the pixel maxima approach, frame boundaries are smooth, but disturbed fibers appear blurred as shown by the arrows (Media 2). (B) When the dead leaves approach is used, blurring does not occur, but frame boundaries can become noticeable (Media 3). The dead leaves approach is preferred for tissue imaging because probe movement can deform the tissue.

Fig. 4
Fig. 4

Real-time video mosaics (see Media 4) from (A) normal in vivo oral tissue, (B) normal in vivo epidermis, and (C) pre-cancerous ex vivo cervical tissue. These results show that the algorithm correctly aligns and combines morphological features of several cell types and anatomical locations. In (A) the distribution of normal squamous epithelial cells is seen in the gingiva and in (B, left) features such as hair and keratin sheaths are visible. An image mosaic (B, right) from a benchtop confocal microscope obtained during the same imaging session reveals similar morphologic features as (B, left). The normal/abnormal junction of ex vivo cervical tissue is shown (C), where the white dotted line indicates the path of the mosaic. Lugol’s iodine was used to help visualize the extent of abnormal tissue; dark brown staining indicates normal mucosa. Corresponding histopathology from inside and outside the normal region is shown (images labeled “Normal” and “CIN II,” respectively). The mosaic also shows that it can be useful to display a large mosaic even with misregistered/blurred frames (white arrows).

Tables (1)

Tables Icon

Table 1 Time Requirements for Video Mosaicing Algorithm

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

r fg ( x i , y i )=  u,v F( u,v ) G * ( u,v )exp[ i2π( u x i M + v y i N ) ],

Metrics