Abstract

Fluorescence imaging can reveal functional, anatomical or pathological features of high interest in medical interventions. We present a novel method to record and display in video rate multispectral color and fluorescence images over the visible and near infrared range. The fast acquisition in multiple channels is achieved through a combination of spectral and temporal multiplexing in a system with two standard color sensors. Accurate color reproduction and high fluorescence unmixing performance are experimentally demonstrated with a prototype system in a challenging imaging scenario. Through spectral simulation and optimization we show that the system is sensitive to all dyes emitting in the visible and near infrared region without changing filters and that the SNR of multiple unmixed components can be kept high if parameters are chosen well. We propose a sensitive per-pixel metric of unmixing quality in a single image based on noise propagation and present a method to visualize the high-dimensional data in a 2D graph, where up to three fluorescent components can be distinguished and segmented.

© 2017 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Color reproduction from low-SNR multispectral images using spatio-spectral Wiener estimation

Yuri Murakami, Ken Fukura, Masahiro Yamaguchi, and Nagaaki Ohyama
Opt. Express 16(6) 4106-4120 (2008)

Dual-mode laparoscopic fluorescence image-guided surgery using a single camera

Daniel C. Gray, Evgenia M. Kim, Victoria E. Cotero, Anshika Bajaj, V. Paul Staudinger, Cristina A. Tan Hehir, and Siavash Yazdanfar
Biomed. Opt. Express 3(8) 1880-1890 (2012)

Medical diagnostic system based on simultaneous multispectral fluorescence imaging

Stefan Andersson-Engels, Jonas Johansson, and Sune Svanberg
Appl. Opt. 33(34) 8022-8029 (1994)

References

  • View by:
  • |
  • |
  • |

  1. N. Sanai and M. S. Berger, “Glioma extent of resection and its impact on patient outcome,” Neurosurgery 62, 753–764 (2008).
    [Crossref] [PubMed]
  2. W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7, 392–401 (2006).
    [Crossref] [PubMed]
  3. M. Kriegmair, R. Baumgartner, R. Knüchel, H. Stepp, F. Hofstädter, and A. Hofstetter, “Detection of early bladder cancer by 5-aminolevulinic acid induced porphyrin fluorescence,” J. Urol. 155, 105–110 (1996).
    [Crossref] [PubMed]
  4. D. Jocham, H. Stepp, and R. Waidelich, “Photodynamic diagnosis in urology: state-of-the-art,” Eur. Urol. 53, 1138–1150 (2008).
    [Crossref]
  5. Q. T. Nguyen and R. Y. Tsien, “Fluorescence-guided surgery with live molecular navigation-a new cutting edge,” Nat. Rev. Cancer 13, 653–662 (2013).
    [Crossref] [PubMed]
  6. A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, C. J. H. van de Velde, and J. V. Frangioni, “Image-guided cancer surgery using near-infrared fluorescence,” Nat. Rev. Clin. Oncol. 10, 507–518 (2013).
    [Crossref] [PubMed]
  7. M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
    [Crossref] [PubMed]
  8. P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012).
    [Crossref] [PubMed]
  9. K. M. Tichauer, K. S. Samkoe, K. J. Sexton, J. R. Gunn, and B. W. Pogue, “Improved tumor contrast achieved by single time point dual-reporter fluorescence imaging,” J. Biomed. Opt. 17, 066001 (2012).
    [Crossref] [PubMed]
  10. N. Dimitriadis, B. Grychtol, L. Maertins, T. Behr, G. Themelis, and N. C. Deliolanis, “Simultaneous real-time multicomponent fluorescence and reflectance imaging method for fluorescence-guided surgery,” Opt. Lett. 41, 1173 (2016).
    [Crossref] [PubMed]
  11. CIE, “Colorimetry - part 2: CIE standard illuminants CIE S 014-2/E:2008 (ISO 11664-2:2007),” Tech. rep., International Organization for Standardization (2008).
  12. M. R. Luo, G. Cui, and B. Rigg, “The development of the CIE 2000 colour-difference formula: CIEDE2000,” Color Res. Appl. 26, 340–350 (2001).
    [Crossref]
  13. S. B. Mondal, S. Gao, N. Zhu, R. Liang, V. Gruev, and S. Achilefu, “Real-time fluorescence image-guided oncologic surgery,” Adv. Cancer Res. 124, 171–211 (2014).
    [Crossref] [PubMed]
  14. R. J. Barlow, Statistics: A Guide to the Use of Statistical Methods in the Physical Sciences (John Wiley & Sons, 2008).
  15. J. Glatz, N. C. Deliolanis, A. Buehler, D. Razansky, and V. Ntziachristos, “Blind source unmixing in multi-spectral optoacoustic tomography,” Opt. Express 19, 3175–3184 (2011).
    [Crossref] [PubMed]
  16. A. Hyvärinen, J. Karhunen, and E. Oja, “Independent component analysis,” Analysis 26, 505 (2002).
  17. R. A. Neher, M. Mitkovski, F. Kirchhoff, E. Neher, F. J. Theis, and A. Zeug, “Blind source separation techniques for the decomposition of multiply labeled fluorescence images,” Biophys. J. 96, 3791–3800 (2009).
    [Crossref] [PubMed]
  18. F. Fereidouni, A. N. Bader, and H. C. Gerritsen, “Spectral phasor analysis allows rapid and reliable unmixing of fluorescence microscopy spectral images,” Opt. Express 20, 12729 (2012).
    [Crossref] [PubMed]

2016 (1)

2014 (1)

S. B. Mondal, S. Gao, N. Zhu, R. Liang, V. Gruev, and S. Achilefu, “Real-time fluorescence image-guided oncologic surgery,” Adv. Cancer Res. 124, 171–211 (2014).
[Crossref] [PubMed]

2013 (2)

Q. T. Nguyen and R. Y. Tsien, “Fluorescence-guided surgery with live molecular navigation-a new cutting edge,” Nat. Rev. Cancer 13, 653–662 (2013).
[Crossref] [PubMed]

A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, C. J. H. van de Velde, and J. V. Frangioni, “Image-guided cancer surgery using near-infrared fluorescence,” Nat. Rev. Clin. Oncol. 10, 507–518 (2013).
[Crossref] [PubMed]

2012 (3)

P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012).
[Crossref] [PubMed]

K. M. Tichauer, K. S. Samkoe, K. J. Sexton, J. R. Gunn, and B. W. Pogue, “Improved tumor contrast achieved by single time point dual-reporter fluorescence imaging,” J. Biomed. Opt. 17, 066001 (2012).
[Crossref] [PubMed]

F. Fereidouni, A. N. Bader, and H. C. Gerritsen, “Spectral phasor analysis allows rapid and reliable unmixing of fluorescence microscopy spectral images,” Opt. Express 20, 12729 (2012).
[Crossref] [PubMed]

2011 (2)

J. Glatz, N. C. Deliolanis, A. Buehler, D. Razansky, and V. Ntziachristos, “Blind source unmixing in multi-spectral optoacoustic tomography,” Opt. Express 19, 3175–3184 (2011).
[Crossref] [PubMed]

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

2009 (1)

R. A. Neher, M. Mitkovski, F. Kirchhoff, E. Neher, F. J. Theis, and A. Zeug, “Blind source separation techniques for the decomposition of multiply labeled fluorescence images,” Biophys. J. 96, 3791–3800 (2009).
[Crossref] [PubMed]

2008 (2)

D. Jocham, H. Stepp, and R. Waidelich, “Photodynamic diagnosis in urology: state-of-the-art,” Eur. Urol. 53, 1138–1150 (2008).
[Crossref]

N. Sanai and M. S. Berger, “Glioma extent of resection and its impact on patient outcome,” Neurosurgery 62, 753–764 (2008).
[Crossref] [PubMed]

2006 (1)

W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7, 392–401 (2006).
[Crossref] [PubMed]

2002 (1)

A. Hyvärinen, J. Karhunen, and E. Oja, “Independent component analysis,” Analysis 26, 505 (2002).

2001 (1)

M. R. Luo, G. Cui, and B. Rigg, “The development of the CIE 2000 colour-difference formula: CIEDE2000,” Color Res. Appl. 26, 340–350 (2001).
[Crossref]

1996 (1)

M. Kriegmair, R. Baumgartner, R. Knüchel, H. Stepp, F. Hofstädter, and A. Hofstetter, “Detection of early bladder cancer by 5-aminolevulinic acid induced porphyrin fluorescence,” J. Urol. 155, 105–110 (1996).
[Crossref] [PubMed]

Achilefu, S.

S. B. Mondal, S. Gao, N. Zhu, R. Liang, V. Gruev, and S. Achilefu, “Real-time fluorescence image-guided oncologic surgery,” Adv. Cancer Res. 124, 171–211 (2014).
[Crossref] [PubMed]

Bader, A. N.

Barlow, R. J.

R. J. Barlow, Statistics: A Guide to the Use of Statistical Methods in the Physical Sciences (John Wiley & Sons, 2008).

Baumgartner, R.

M. Kriegmair, R. Baumgartner, R. Knüchel, H. Stepp, F. Hofstädter, and A. Hofstetter, “Detection of early bladder cancer by 5-aminolevulinic acid induced porphyrin fluorescence,” J. Urol. 155, 105–110 (1996).
[Crossref] [PubMed]

Behr, T.

Berger, M. S.

N. Sanai and M. S. Berger, “Glioma extent of resection and its impact on patient outcome,” Neurosurgery 62, 753–764 (2008).
[Crossref] [PubMed]

Buehler, A.

Crisp, J. L.

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

Cui, G.

M. R. Luo, G. Cui, and B. Rigg, “The development of the CIE 2000 colour-difference formula: CIEDE2000,” Color Res. Appl. 26, 340–350 (2001).
[Crossref]

Deliolanis, N. C.

Dimitriadis, N.

Fereidouni, F.

Frangioni, J. V.

A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, C. J. H. van de Velde, and J. V. Frangioni, “Image-guided cancer surgery using near-infrared fluorescence,” Nat. Rev. Clin. Oncol. 10, 507–518 (2013).
[Crossref] [PubMed]

Friedman, B.

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

Gao, S.

S. B. Mondal, S. Gao, N. Zhu, R. Liang, V. Gruev, and S. Achilefu, “Real-time fluorescence image-guided oncologic surgery,” Adv. Cancer Res. 124, 171–211 (2014).
[Crossref] [PubMed]

Gerritsen, H. C.

Glatz, J.

Gross, L. A.

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

Gruev, V.

S. B. Mondal, S. Gao, N. Zhu, R. Liang, V. Gruev, and S. Achilefu, “Real-time fluorescence image-guided oncologic surgery,” Adv. Cancer Res. 124, 171–211 (2014).
[Crossref] [PubMed]

Grychtol, B.

Gunn, J. R.

K. M. Tichauer, K. S. Samkoe, K. J. Sexton, J. R. Gunn, and B. W. Pogue, “Improved tumor contrast achieved by single time point dual-reporter fluorescence imaging,” J. Biomed. Opt. 17, 066001 (2012).
[Crossref] [PubMed]

Hofstädter, F.

M. Kriegmair, R. Baumgartner, R. Knüchel, H. Stepp, F. Hofstädter, and A. Hofstetter, “Detection of early bladder cancer by 5-aminolevulinic acid induced porphyrin fluorescence,” J. Urol. 155, 105–110 (1996).
[Crossref] [PubMed]

Hofstetter, A.

M. Kriegmair, R. Baumgartner, R. Knüchel, H. Stepp, F. Hofstädter, and A. Hofstetter, “Detection of early bladder cancer by 5-aminolevulinic acid induced porphyrin fluorescence,” J. Urol. 155, 105–110 (1996).
[Crossref] [PubMed]

Hutteman, M.

A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, C. J. H. van de Velde, and J. V. Frangioni, “Image-guided cancer surgery using near-infrared fluorescence,” Nat. Rev. Clin. Oncol. 10, 507–518 (2013).
[Crossref] [PubMed]

Hyvärinen, A.

A. Hyvärinen, J. Karhunen, and E. Oja, “Independent component analysis,” Analysis 26, 505 (2002).

Jacobs, V. L.

P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012).
[Crossref] [PubMed]

Jocham, D.

D. Jocham, H. Stepp, and R. Waidelich, “Photodynamic diagnosis in urology: state-of-the-art,” Eur. Urol. 53, 1138–1150 (2008).
[Crossref]

Karhunen, J.

A. Hyvärinen, J. Karhunen, and E. Oja, “Independent component analysis,” Analysis 26, 505 (2002).

Kirchhoff, F.

R. A. Neher, M. Mitkovski, F. Kirchhoff, E. Neher, F. J. Theis, and A. Zeug, “Blind source separation techniques for the decomposition of multiply labeled fluorescence images,” Biophys. J. 96, 3791–3800 (2009).
[Crossref] [PubMed]

Knüchel, R.

M. Kriegmair, R. Baumgartner, R. Knüchel, H. Stepp, F. Hofstädter, and A. Hofstetter, “Detection of early bladder cancer by 5-aminolevulinic acid induced porphyrin fluorescence,” J. Urol. 155, 105–110 (1996).
[Crossref] [PubMed]

Kriegmair, M.

M. Kriegmair, R. Baumgartner, R. Knüchel, H. Stepp, F. Hofstädter, and A. Hofstetter, “Detection of early bladder cancer by 5-aminolevulinic acid induced porphyrin fluorescence,” J. Urol. 155, 105–110 (1996).
[Crossref] [PubMed]

Leblond, F.

P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012).
[Crossref] [PubMed]

Liang, R.

S. B. Mondal, S. Gao, N. Zhu, R. Liang, V. Gruev, and S. Achilefu, “Real-time fluorescence image-guided oncologic surgery,” Adv. Cancer Res. 124, 171–211 (2014).
[Crossref] [PubMed]

Luo, M. R.

M. R. Luo, G. Cui, and B. Rigg, “The development of the CIE 2000 colour-difference formula: CIEDE2000,” Color Res. Appl. 26, 340–350 (2001).
[Crossref]

Maertins, L.

Meinel, T.

W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7, 392–401 (2006).
[Crossref] [PubMed]

Mitkovski, M.

R. A. Neher, M. Mitkovski, F. Kirchhoff, E. Neher, F. J. Theis, and A. Zeug, “Blind source separation techniques for the decomposition of multiply labeled fluorescence images,” Biophys. J. 96, 3791–3800 (2009).
[Crossref] [PubMed]

Mondal, S. B.

S. B. Mondal, S. Gao, N. Zhu, R. Liang, V. Gruev, and S. Achilefu, “Real-time fluorescence image-guided oncologic surgery,” Adv. Cancer Res. 124, 171–211 (2014).
[Crossref] [PubMed]

Neher, E.

R. A. Neher, M. Mitkovski, F. Kirchhoff, E. Neher, F. J. Theis, and A. Zeug, “Blind source separation techniques for the decomposition of multiply labeled fluorescence images,” Biophys. J. 96, 3791–3800 (2009).
[Crossref] [PubMed]

Neher, R. A.

R. A. Neher, M. Mitkovski, F. Kirchhoff, E. Neher, F. J. Theis, and A. Zeug, “Blind source separation techniques for the decomposition of multiply labeled fluorescence images,” Biophys. J. 96, 3791–3800 (2009).
[Crossref] [PubMed]

Nguyen, L. T.

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

Nguyen, Q. T.

Q. T. Nguyen and R. Y. Tsien, “Fluorescence-guided surgery with live molecular navigation-a new cutting edge,” Nat. Rev. Cancer 13, 653–662 (2013).
[Crossref] [PubMed]

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

Ntziachristos, V.

Oja, E.

A. Hyvärinen, J. Karhunen, and E. Oja, “Independent component analysis,” Analysis 26, 505 (2002).

Paulsen, K. D.

P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012).
[Crossref] [PubMed]

Pichlmeier, U.

W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7, 392–401 (2006).
[Crossref] [PubMed]

Pogue, B. W.

K. M. Tichauer, K. S. Samkoe, K. J. Sexton, J. R. Gunn, and B. W. Pogue, “Improved tumor contrast achieved by single time point dual-reporter fluorescence imaging,” J. Biomed. Opt. 17, 066001 (2012).
[Crossref] [PubMed]

Razansky, D.

Reulen, H. J.

W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7, 392–401 (2006).
[Crossref] [PubMed]

Rigg, B.

M. R. Luo, G. Cui, and B. Rigg, “The development of the CIE 2000 colour-difference formula: CIEDE2000,” Color Res. Appl. 26, 340–350 (2001).
[Crossref]

Roberts, D. W.

P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012).
[Crossref] [PubMed]

Samkoe, K. S.

K. M. Tichauer, K. S. Samkoe, K. J. Sexton, J. R. Gunn, and B. W. Pogue, “Improved tumor contrast achieved by single time point dual-reporter fluorescence imaging,” J. Biomed. Opt. 17, 066001 (2012).
[Crossref] [PubMed]

Sanai, N.

N. Sanai and M. S. Berger, “Glioma extent of resection and its impact on patient outcome,” Neurosurgery 62, 753–764 (2008).
[Crossref] [PubMed]

Sexton, K. J.

K. M. Tichauer, K. S. Samkoe, K. J. Sexton, J. R. Gunn, and B. W. Pogue, “Improved tumor contrast achieved by single time point dual-reporter fluorescence imaging,” J. Biomed. Opt. 17, 066001 (2012).
[Crossref] [PubMed]

Steinbach, P.

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

Stepp, H.

D. Jocham, H. Stepp, and R. Waidelich, “Photodynamic diagnosis in urology: state-of-the-art,” Eur. Urol. 53, 1138–1150 (2008).
[Crossref]

M. Kriegmair, R. Baumgartner, R. Knüchel, H. Stepp, F. Hofstädter, and A. Hofstetter, “Detection of early bladder cancer by 5-aminolevulinic acid induced porphyrin fluorescence,” J. Urol. 155, 105–110 (1996).
[Crossref] [PubMed]

Stummer, W.

W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7, 392–401 (2006).
[Crossref] [PubMed]

Theis, F. J.

R. A. Neher, M. Mitkovski, F. Kirchhoff, E. Neher, F. J. Theis, and A. Zeug, “Blind source separation techniques for the decomposition of multiply labeled fluorescence images,” Biophys. J. 96, 3791–3800 (2009).
[Crossref] [PubMed]

Themelis, G.

Tichauer, K. M.

K. M. Tichauer, K. S. Samkoe, K. J. Sexton, J. R. Gunn, and B. W. Pogue, “Improved tumor contrast achieved by single time point dual-reporter fluorescence imaging,” J. Biomed. Opt. 17, 066001 (2012).
[Crossref] [PubMed]

Tsien, R. Y.

Q. T. Nguyen and R. Y. Tsien, “Fluorescence-guided surgery with live molecular navigation-a new cutting edge,” Nat. Rev. Cancer 13, 653–662 (2013).
[Crossref] [PubMed]

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

Vahrmeijer, A. L.

A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, C. J. H. van de Velde, and J. V. Frangioni, “Image-guided cancer surgery using near-infrared fluorescence,” Nat. Rev. Clin. Oncol. 10, 507–518 (2013).
[Crossref] [PubMed]

Valdés, P. A.

P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012).
[Crossref] [PubMed]

van de Velde, C. J. H.

A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, C. J. H. van de Velde, and J. V. Frangioni, “Image-guided cancer surgery using near-infrared fluorescence,” Nat. Rev. Clin. Oncol. 10, 507–518 (2013).
[Crossref] [PubMed]

van der Vorst, J. R.

A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, C. J. H. van de Velde, and J. V. Frangioni, “Image-guided cancer surgery using near-infrared fluorescence,” Nat. Rev. Clin. Oncol. 10, 507–518 (2013).
[Crossref] [PubMed]

Waidelich, R.

D. Jocham, H. Stepp, and R. Waidelich, “Photodynamic diagnosis in urology: state-of-the-art,” Eur. Urol. 53, 1138–1150 (2008).
[Crossref]

Whitney, M. A.

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

Wiestler, O. D.

W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7, 392–401 (2006).
[Crossref] [PubMed]

Wilson, B. C.

P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012).
[Crossref] [PubMed]

Zanella, F.

W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7, 392–401 (2006).
[Crossref] [PubMed]

Zeug, A.

R. A. Neher, M. Mitkovski, F. Kirchhoff, E. Neher, F. J. Theis, and A. Zeug, “Blind source separation techniques for the decomposition of multiply labeled fluorescence images,” Biophys. J. 96, 3791–3800 (2009).
[Crossref] [PubMed]

Zhu, N.

S. B. Mondal, S. Gao, N. Zhu, R. Liang, V. Gruev, and S. Achilefu, “Real-time fluorescence image-guided oncologic surgery,” Adv. Cancer Res. 124, 171–211 (2014).
[Crossref] [PubMed]

Adv. Cancer Res. (1)

S. B. Mondal, S. Gao, N. Zhu, R. Liang, V. Gruev, and S. Achilefu, “Real-time fluorescence image-guided oncologic surgery,” Adv. Cancer Res. 124, 171–211 (2014).
[Crossref] [PubMed]

Analysis (1)

A. Hyvärinen, J. Karhunen, and E. Oja, “Independent component analysis,” Analysis 26, 505 (2002).

Biophys. J. (1)

R. A. Neher, M. Mitkovski, F. Kirchhoff, E. Neher, F. J. Theis, and A. Zeug, “Blind source separation techniques for the decomposition of multiply labeled fluorescence images,” Biophys. J. 96, 3791–3800 (2009).
[Crossref] [PubMed]

Color Res. Appl. (1)

M. R. Luo, G. Cui, and B. Rigg, “The development of the CIE 2000 colour-difference formula: CIEDE2000,” Color Res. Appl. 26, 340–350 (2001).
[Crossref]

Eur. Urol. (1)

D. Jocham, H. Stepp, and R. Waidelich, “Photodynamic diagnosis in urology: state-of-the-art,” Eur. Urol. 53, 1138–1150 (2008).
[Crossref]

J. Biomed. Opt. (1)

K. M. Tichauer, K. S. Samkoe, K. J. Sexton, J. R. Gunn, and B. W. Pogue, “Improved tumor contrast achieved by single time point dual-reporter fluorescence imaging,” J. Biomed. Opt. 17, 066001 (2012).
[Crossref] [PubMed]

J. Urol. (1)

M. Kriegmair, R. Baumgartner, R. Knüchel, H. Stepp, F. Hofstädter, and A. Hofstetter, “Detection of early bladder cancer by 5-aminolevulinic acid induced porphyrin fluorescence,” J. Urol. 155, 105–110 (1996).
[Crossref] [PubMed]

Lancet Oncol. (1)

W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen, “Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial,” Lancet Oncol. 7, 392–401 (2006).
[Crossref] [PubMed]

Nat. Biotechnol. (1)

M. A. Whitney, J. L. Crisp, L. T. Nguyen, B. Friedman, L. A. Gross, P. Steinbach, R. Y. Tsien, and Q. T. Nguyen, “Fluorescent peptides highlight peripheral nerves during surgery in mice,” Nat. Biotechnol. 29, 352–356 (2011).
[Crossref] [PubMed]

Nat. Rev. Cancer (1)

Q. T. Nguyen and R. Y. Tsien, “Fluorescence-guided surgery with live molecular navigation-a new cutting edge,” Nat. Rev. Cancer 13, 653–662 (2013).
[Crossref] [PubMed]

Nat. Rev. Clin. Oncol. (1)

A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, C. J. H. van de Velde, and J. V. Frangioni, “Image-guided cancer surgery using near-infrared fluorescence,” Nat. Rev. Clin. Oncol. 10, 507–518 (2013).
[Crossref] [PubMed]

Neurosurgery (1)

N. Sanai and M. S. Berger, “Glioma extent of resection and its impact on patient outcome,” Neurosurgery 62, 753–764 (2008).
[Crossref] [PubMed]

Opt. Express (2)

Opt. Lett. (1)

Sci. Rep. (1)

P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, “Quantitative, spectrally-resolved intraoperative fluorescence imaging,” Sci. Rep. 2, 798 (2012).
[Crossref] [PubMed]

Other (2)

CIE, “Colorimetry - part 2: CIE standard illuminants CIE S 014-2/E:2008 (ISO 11664-2:2007),” Tech. rep., International Organization for Standardization (2008).

R. J. Barlow, Statistics: A Guide to the Use of Statistical Methods in the Physical Sciences (John Wiley & Sons, 2008).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1

(a) Schematic of the optical setup. The object is illuminated alternately with either of the two spectrally complementary lights LA or LB. OL: objective lens, BS: beam splitter, F1 and F2: complementary multiband filters, L1 and L2: imaging lenses, S1 and S2: color sensors. (b) RGGB pixel pattern of the color sensors. (c) Normalized LED spectra of L1 and (d) normalized LED spectra of L2. (e) Transmission t of multiple bandpass filter F1. (f) Transmission t of multiple bandpass filter F2.(g) Quantum efficiency s of the sensor S1 and (h) of the sensor S2. (i) Spectral and temporal multiplexing scheme. In phase A: LA illuminates the object, S1 (with F1) records the fluorescence image, and S2 (with F2) records the reflectance image. In phase B is the opposite: LB illuminates the object, S1 (with F1) records the complementary reflectance image and S2 (with F2) records the complementary fluorescence image.

Fig. 2
Fig. 2

(a) Uncorrected color image produced by averaging the color channels of the two sensors (for visibility the black and the white tiles are scaled to match the ideal RGB average brightness). (b) Corrected image. Best correction result using linear transformation with an additional offset. (c) Comparison of color reproduction methods for all 30 tiles. For each tile 5 different processing scenarios are shown: (A) Uncorrected colors (average of two sensors, for visibility the black and the white tiles are scaled to match the ideal RGB average brightness). (B) Optimal correction by using combined information from sensors A and B and transformation to xyz color space. (C) Best color correction using only sensor A. (D) Best color correction using only sensor B. (E) Ideal value as perceived under D65 illumination by the human eye. (d) Boxplot of the CIEDE2000 errors ΔE00 after optimal correction using the color information from only sensor A, only sensor B and both sensors. Dotted line at 1 is the absolute distinction limit; dashed line at 3.5 indicates a limit at which color differences can be hardly distinguished.

Fig. 3
Fig. 3

Simulation of system sensitivity. (a) Fluorochrome emission spectrum (Gaussian distribution). (b) Combination of filter spectral transmission and sensor sensitivity. (c) Sensor signal intensity for the six channels calculated from the convolution of (a) and (b). S1 and S2 are simulating monochrome sensors by combining channels R1, G1, B1 and R2, G2, B2, respectively

Fig. 4
Fig. 4

Fluorescent images of the test vials containing the fluorescent dyes Atto532, Atto565 and Atto610. (a) Image of the sum of all channels (one acquisition). Vial labels A – F match with Table 1. (b) Images of each channel for both sensors (only one of the two G pixels is shown; individual acquisition). (c) Excitation spectra (dotted lines) and emission spectra (solid lines) of Atto532 (green), Atto565 (yellow) and Atto610 (red). (d) Spectrally unmixed fluorescent dye components using channels from both sensors (individual acquisition). (f) Standard deviation of 100 sequentially acquired and unmixed fluorescent images (as in (d)). (e) r values to assess the unmixing quality. The region in the white box is magnified for better visibility. (g) Unmixing results using only the sensor channels R1, G1, G1* and B1 of a single image and (h) unmixing results using only the sensor channels R2, G2, G2* and B2 of a single image. Images of (g) and (h) also contain strong negative values, but the scale is limited to 0 as the lower limit. (table) Maximum scaling value of the intensity images.

Fig. 5
Fig. 5

Optimization of filters and fluorescent compounds. (a) Optimal selection of two fluorescent dyes: relative SNR (in dB) of each of two Gaussian dyes and combined |SNR|. (b) Optimal selection of 2 – 6 fluorescent dyes: Visualization of the optimized center wavelengths for the given multiband filters F1 and F2. (c) Optimal design of filters: visualization of the optimized band structure for given dyes Atto532, Atto565 and Atto610 given sensor sensitivity curves, having the number of filter bands as a parameter.

Fig. 6
Fig. 6

Principle component analysis: (a) the 2D histogram of the first two PCA component vectors as basis (thresholded at 1% of the maximum signal). (b) Segmented images of the fluorescent signals (regions A to F as in PCA space shown in (a)). Fluorescent signals unmixed in PCA space spanned by the first two component vectors. Pure dye signals spanning the unmixing triangle in (a) are set by projecting the pure dye signatures onto the first two PCA component vectors.

Tables (4)

Tables Icon

Table 1 Peak absorption (λabs), peak fluorescense (λfl) and concentration of dyes in vials A to F.

Tables Icon

Table 2 Upper part: angle in degrees between the different dye spectral signatures in the dual sensor system (S1 + S2) and the two single sensor systems (sensor S1 or sensor S2). Lower part: the percentage of photons η orginating from each dye and detected by each system.

Tables Icon

Table 3 Optimal choices of peak emission wavelengths of Nf dyes and the corresponding relative SNR values.

Tables Icon

Table 4 Relative SNR in dB of the optimization results varying the alignment of the filter bands for the two sensors (number of bands Nb as a fixed parameter) for the algorithm that performed best (PS: PatternSearch, GS: GlobalSearch, GA: Genetic Algorithm). The reported filter bands characterize the optimal transmission bands (bands reported alternating between F1 and F2).

Equations (19)

Equations on this page are rendered with MathJax. Learn more.

e i = c = 1 6 K i , c d c + K 0 .
d exp K i , c and K 0 e exp D 65 illumination f exp
Δ E 00 , tile = Δ E 00 ( f tile exp , f tile D 65 , human eye )
arg min K i , c , K 0 ( 1 30 tile = 1 30 Δ E 00 , tile ( d tile exp K i , c , K 0 f tile exp ; f tile D 65 , human eye ) )
d = Mi ,
i ˜ = M + d = Ud .
Σ i ˜ = U Σ d U ,
Σ i ˜ j , j = c = 1 N c U j , c 2 Σ d c , c = c = 1 N c U j , c 2 σ d c 2
( diag ( Σ i ˜ ) ) 1 2
SNR = ( diag ( Σ i ˜ ) ) 1 2 i ˜ .
d ˜ = M i ˜ = MUd .
Σ d ˜ = M Σ i ˜ M = MU Σ d U M ,
Σ r = Σ d + MU Σ d U M MU Σ d Σ d U M .
r = ( diag ( Σ r ) ) 1 2 r .
σ d c 2 = σ d c Poisson 2 + σ 0 2 .
σ i ˜ f 2 = Σ i ˜ f , f c = 1 N c U f , c 2 d c = c = 1 N c φ = 1 N f U f , c 2 M c , φ i φ .
| SNR | = ( 1 N f f N f SNR f 2 ) 1 2 .
q = [ p 1 , p 2 ] n
[ S A , 1 S B , 1 S C , 1 S A , 2 S B , 2 S C , 2 1 1 1 ] i = [ q 1 q 2 1 ] ( c d c ) .

Metrics