Abstract

Fluorescence widely coexists with reflectance in the real world, and an accurate representation of these two components in a scene is vitally important. Despite the rich knowledge of fluorescence mechanisms and behaviors, traditional fluorescence imaging approaches are quite limited in efficiency and quality. To address these two shortcomings, we propose a bispectral coding scheme to capture fluorescence and reflectance: multiplexing code is applied to excitation spectrums to raise the signal-to-noise ratio, and compressive sampling code is applied to emission spectrums for high efficiency. For computational reconstruction from the sparse coded measurements, the redundancy in both components promises recovery from sparse measurements, and the difference between their redundancies promises accurate separation. Mathematically, we cast the reconstruction as a joint optimization, whose solution can be derived by the Augmented Lagrange Method. In our experiment, results on both synthetic data and real data captured by our prototype validate the proposed approach, and we also demonstrate its advantages in two computer vision tasks—photorealistic relighting and segmentation.

© 2014 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. R. Donaldson, “Spectrophotometry of fluorescent pigments,” Br. J. Appl. Phys. 5(6), 210–214 (1954).
    [CrossRef]
  2. I. Sato, C. Zhang, “Image-based separation of reflective and fluorescent components using illumination variant and invariant color,” IEEE Trans. Pattern Anal. 35(12), 2866–2877 (2013).
    [CrossRef]
  3. A. D. McNaught, A. Wilkinson, Compendium of Chemical Terminology (Blackwell Science, 1997).
  4. A. Springsteen, “Introduction to measurement of color of fluorescent materials,” Anal. Chim. Acta 380(2), 183–192 (1999).
    [CrossRef]
  5. G. M. Johnson, M. D. Fairchild, “Full-spectral color calculations in realistic image synthesis,” IEEE Comput. Graphics Appl. 19(4), 47–53 (1999).
    [CrossRef]
  6. M. B. Hullin, J. Hanika, B. Ajdin, H.-P. Seidel, J. Kautz, H. P. A. Lensch, “Acquisition and analysis of bis-pectral bidirectional reflectance and reradiation distribution functions,” ACM Trans. Graphics 29(4), 1–7 (2010).
    [CrossRef]
  7. M. Soriano, W. Oblefias, C. Saloma, “Fluorescence spectrum estimation using multiple color images and minimum negativity constraint,” Opt. Express 10(25), 1458–1464 (2002).
    [CrossRef] [PubMed]
  8. Q. Liu, K. Chen, M. Martin, A. Wintenberg, R. Lenarduzzi, M. Panjehpour, B. F. Overholt, T. Vo-Dinh, “Development of a synchronous fluorescence imaging system and data analysis methods,” Opt. Express 15(20), 12583–12594 (2007).
    [CrossRef] [PubMed]
  9. T. Vo-Dinh, “Principle of synchronous luminescence (SL) technique for biomedical diagnostics,” Proc. SPIE 3911, 42–49 (2000).
    [CrossRef]
  10. I. Sato, T. Okabe, Y. Sato, “Bispectral photometric stereo based on fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 270–277.
  11. S. Han, Y. Matsushita, I. Sato, T. Okabe, Y. Sato, “Camera spectral sensitivity estimation from a single image under unknown illumination by using fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 805–812.
  12. C. Chi, H. Yoo, M. Ben-Ezra, “Multi-spectral imaging by optimized wide band illumination,” Int. J. Comput. Vision 86(2–3), 140–151 (2010).
    [CrossRef]
  13. M. Alterman, Y. Schechner, A. Weiss, “Multiplexed fluorescence unmixing,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2010), pp. 1–8.
  14. J. Park, M. Lee, M. D. Grossberg, S. K. Nayar, “Multispectral imaging using multiplexed illumination,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2007), pp. 1–8.
  15. S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Separating the fluorescence and reflectance components of coral spectra,” Appl. Opt. 40(21), 3614–3621 (2001).
    [CrossRef]
  16. M. E. Gehm, R. John, D. J. Brady, R. M. Willett, T. J. Schulz, “Single-shot compressive spectral imaging with a dual disperser architecture,” Opt. Express 15(21), 14013–14027 (2007).
    [CrossRef] [PubMed]
  17. A. Wagadarikar, R. John, R. Willett, D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt. 47(10), B44–B51 (2008).
    [CrossRef] [PubMed]
  18. R. Horisaki, X. Xiao, J. Tanida, B. Javidi, “Feasibility study for compressive multi-dimensional integral imaging,” Opt. Express 21(4), 4263–4279 (2013).
    [CrossRef] [PubMed]
  19. R. Horisaki, J. Tanida, “Multi-channel data acquisition using multiplexed imaging with spatial encoding,” Opt. Express 18(22), 23041–23053 (2010).
    [CrossRef] [PubMed]
  20. Y. Wu, I. O. Mirza, G. R. Arce, D. W. Prather, “Development of a digital-micromirror-device-based multishot snapshot spectral imaging system,” Opt. Lett. 36(14), 2692–2694 (2011).
    [CrossRef] [PubMed]
  21. Y. August, A. Stern, “Compressive sensing spectrometry based on liquid crystal devices,” Opt. Lett. 38(23), 4996–4999 (2013).
    [CrossRef] [PubMed]
  22. G. Wetzstein, I. Ihrke, W. Heidrich, “On plenoptic multiplexing and reconstruction,” Int. J. Comput. Vision 101(2), 384–400 (2013).
    [CrossRef]
  23. N. Ratner, Y. Y. Schechner, “Illumination multiplexing within fundamental limits,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2011), pp. 1–8.
  24. Y. Y. Schechner, S. K. Nayar, P. N. Belhumeur, “Multiplexing for optimal lighting,” IEEE Pattern Anal. Mach. Intell. 29(8), 1339–1354 (2007).
    [CrossRef]
  25. C. Chen, D. Vaquero, M. Turk, “Illumination demultiplexing from a single image,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2011), pp. 17–24.
  26. F. Moreno-Noguer, S. Nayar, P. Belhumeur, “Optimal illumination for image and video relighting,” in Proceedings of IEE European Conference on Visual Media Production (IEE, 2005), pp. 201–210.
  27. S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learning 3(1), 741–755 (2010).
    [CrossRef]
  28. A. Yang, S. Sastry, A. Ganesh, Y. Ma, “Fast-minimization algorithms and an application in robust face recognition: A review,” in Proceedings of IEEE Conference on Image Processing (IEEE, 2010), pp. 1849–1852.
  29. Y. Deng, Q. Dai, Z. Zhang, “An overview of computational sparse models and their applications in artificial intelligence,” Artif. Intell. Evol. Comput. Metaheuristics 427, 345–369 (2012).
    [CrossRef]
  30. Z. Lin, M. Chen, L. Wu, Y. Ma, “The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices,” in Technical Report UILU-ENG-09-2215 (UIUC, 2009).
  31. M. Harwit, N. J. A. Sloane, Hadamard Transform Optics (Academic, 1979).
  32. A. Lam, I. Sato, “Spectral modeling and relighting of reflective-fluorescent scenes,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2013), pp. 1452–1459.

2013 (4)

I. Sato, C. Zhang, “Image-based separation of reflective and fluorescent components using illumination variant and invariant color,” IEEE Trans. Pattern Anal. 35(12), 2866–2877 (2013).
[CrossRef]

G. Wetzstein, I. Ihrke, W. Heidrich, “On plenoptic multiplexing and reconstruction,” Int. J. Comput. Vision 101(2), 384–400 (2013).
[CrossRef]

R. Horisaki, X. Xiao, J. Tanida, B. Javidi, “Feasibility study for compressive multi-dimensional integral imaging,” Opt. Express 21(4), 4263–4279 (2013).
[CrossRef] [PubMed]

Y. August, A. Stern, “Compressive sensing spectrometry based on liquid crystal devices,” Opt. Lett. 38(23), 4996–4999 (2013).
[CrossRef] [PubMed]

2012 (1)

Y. Deng, Q. Dai, Z. Zhang, “An overview of computational sparse models and their applications in artificial intelligence,” Artif. Intell. Evol. Comput. Metaheuristics 427, 345–369 (2012).
[CrossRef]

2011 (1)

2010 (4)

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learning 3(1), 741–755 (2010).
[CrossRef]

R. Horisaki, J. Tanida, “Multi-channel data acquisition using multiplexed imaging with spatial encoding,” Opt. Express 18(22), 23041–23053 (2010).
[CrossRef] [PubMed]

C. Chi, H. Yoo, M. Ben-Ezra, “Multi-spectral imaging by optimized wide band illumination,” Int. J. Comput. Vision 86(2–3), 140–151 (2010).
[CrossRef]

M. B. Hullin, J. Hanika, B. Ajdin, H.-P. Seidel, J. Kautz, H. P. A. Lensch, “Acquisition and analysis of bis-pectral bidirectional reflectance and reradiation distribution functions,” ACM Trans. Graphics 29(4), 1–7 (2010).
[CrossRef]

2008 (1)

2007 (3)

2002 (1)

2001 (1)

2000 (1)

T. Vo-Dinh, “Principle of synchronous luminescence (SL) technique for biomedical diagnostics,” Proc. SPIE 3911, 42–49 (2000).
[CrossRef]

1999 (2)

A. Springsteen, “Introduction to measurement of color of fluorescent materials,” Anal. Chim. Acta 380(2), 183–192 (1999).
[CrossRef]

G. M. Johnson, M. D. Fairchild, “Full-spectral color calculations in realistic image synthesis,” IEEE Comput. Graphics Appl. 19(4), 47–53 (1999).
[CrossRef]

1954 (1)

R. Donaldson, “Spectrophotometry of fluorescent pigments,” Br. J. Appl. Phys. 5(6), 210–214 (1954).
[CrossRef]

Ajdin, B.

M. B. Hullin, J. Hanika, B. Ajdin, H.-P. Seidel, J. Kautz, H. P. A. Lensch, “Acquisition and analysis of bis-pectral bidirectional reflectance and reradiation distribution functions,” ACM Trans. Graphics 29(4), 1–7 (2010).
[CrossRef]

Alterman, M.

M. Alterman, Y. Schechner, A. Weiss, “Multiplexed fluorescence unmixing,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2010), pp. 1–8.

Arce, G. R.

August, Y.

Belhumeur, P.

F. Moreno-Noguer, S. Nayar, P. Belhumeur, “Optimal illumination for image and video relighting,” in Proceedings of IEE European Conference on Visual Media Production (IEE, 2005), pp. 201–210.

Belhumeur, P. N.

Y. Y. Schechner, S. K. Nayar, P. N. Belhumeur, “Multiplexing for optimal lighting,” IEEE Pattern Anal. Mach. Intell. 29(8), 1339–1354 (2007).
[CrossRef]

Ben-Ezra, M.

C. Chi, H. Yoo, M. Ben-Ezra, “Multi-spectral imaging by optimized wide band illumination,” Int. J. Comput. Vision 86(2–3), 140–151 (2010).
[CrossRef]

Boyd, S.

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learning 3(1), 741–755 (2010).
[CrossRef]

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Separating the fluorescence and reflectance components of coral spectra,” Appl. Opt. 40(21), 3614–3621 (2001).
[CrossRef]

Brady, D.

Brady, D. J.

Chen, C.

C. Chen, D. Vaquero, M. Turk, “Illumination demultiplexing from a single image,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2011), pp. 17–24.

Chen, K.

Chen, M.

Z. Lin, M. Chen, L. Wu, Y. Ma, “The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices,” in Technical Report UILU-ENG-09-2215 (UIUC, 2009).

Chi, C.

C. Chi, H. Yoo, M. Ben-Ezra, “Multi-spectral imaging by optimized wide band illumination,” Int. J. Comput. Vision 86(2–3), 140–151 (2010).
[CrossRef]

Chu, E.

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learning 3(1), 741–755 (2010).
[CrossRef]

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Separating the fluorescence and reflectance components of coral spectra,” Appl. Opt. 40(21), 3614–3621 (2001).
[CrossRef]

Dai, Q.

Y. Deng, Q. Dai, Z. Zhang, “An overview of computational sparse models and their applications in artificial intelligence,” Artif. Intell. Evol. Comput. Metaheuristics 427, 345–369 (2012).
[CrossRef]

Deng, Y.

Y. Deng, Q. Dai, Z. Zhang, “An overview of computational sparse models and their applications in artificial intelligence,” Artif. Intell. Evol. Comput. Metaheuristics 427, 345–369 (2012).
[CrossRef]

Donaldson, R.

R. Donaldson, “Spectrophotometry of fluorescent pigments,” Br. J. Appl. Phys. 5(6), 210–214 (1954).
[CrossRef]

Eckstein, J.

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learning 3(1), 741–755 (2010).
[CrossRef]

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Separating the fluorescence and reflectance components of coral spectra,” Appl. Opt. 40(21), 3614–3621 (2001).
[CrossRef]

Fairchild, M. D.

G. M. Johnson, M. D. Fairchild, “Full-spectral color calculations in realistic image synthesis,” IEEE Comput. Graphics Appl. 19(4), 47–53 (1999).
[CrossRef]

Ganesh, A.

A. Yang, S. Sastry, A. Ganesh, Y. Ma, “Fast-minimization algorithms and an application in robust face recognition: A review,” in Proceedings of IEEE Conference on Image Processing (IEEE, 2010), pp. 1849–1852.

Gehm, M. E.

Grossberg, M. D.

J. Park, M. Lee, M. D. Grossberg, S. K. Nayar, “Multispectral imaging using multiplexed illumination,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2007), pp. 1–8.

Han, S.

S. Han, Y. Matsushita, I. Sato, T. Okabe, Y. Sato, “Camera spectral sensitivity estimation from a single image under unknown illumination by using fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 805–812.

Hanika, J.

M. B. Hullin, J. Hanika, B. Ajdin, H.-P. Seidel, J. Kautz, H. P. A. Lensch, “Acquisition and analysis of bis-pectral bidirectional reflectance and reradiation distribution functions,” ACM Trans. Graphics 29(4), 1–7 (2010).
[CrossRef]

Harwit, M.

M. Harwit, N. J. A. Sloane, Hadamard Transform Optics (Academic, 1979).

Heidrich, W.

G. Wetzstein, I. Ihrke, W. Heidrich, “On plenoptic multiplexing and reconstruction,” Int. J. Comput. Vision 101(2), 384–400 (2013).
[CrossRef]

Horisaki, R.

Hullin, M. B.

M. B. Hullin, J. Hanika, B. Ajdin, H.-P. Seidel, J. Kautz, H. P. A. Lensch, “Acquisition and analysis of bis-pectral bidirectional reflectance and reradiation distribution functions,” ACM Trans. Graphics 29(4), 1–7 (2010).
[CrossRef]

Ihrke, I.

G. Wetzstein, I. Ihrke, W. Heidrich, “On plenoptic multiplexing and reconstruction,” Int. J. Comput. Vision 101(2), 384–400 (2013).
[CrossRef]

Javidi, B.

John, R.

Johnson, G. M.

G. M. Johnson, M. D. Fairchild, “Full-spectral color calculations in realistic image synthesis,” IEEE Comput. Graphics Appl. 19(4), 47–53 (1999).
[CrossRef]

Kautz, J.

M. B. Hullin, J. Hanika, B. Ajdin, H.-P. Seidel, J. Kautz, H. P. A. Lensch, “Acquisition and analysis of bis-pectral bidirectional reflectance and reradiation distribution functions,” ACM Trans. Graphics 29(4), 1–7 (2010).
[CrossRef]

Lam, A.

A. Lam, I. Sato, “Spectral modeling and relighting of reflective-fluorescent scenes,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2013), pp. 1452–1459.

Lee, M.

J. Park, M. Lee, M. D. Grossberg, S. K. Nayar, “Multispectral imaging using multiplexed illumination,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2007), pp. 1–8.

Lenarduzzi, R.

Lensch, H. P. A.

M. B. Hullin, J. Hanika, B. Ajdin, H.-P. Seidel, J. Kautz, H. P. A. Lensch, “Acquisition and analysis of bis-pectral bidirectional reflectance and reradiation distribution functions,” ACM Trans. Graphics 29(4), 1–7 (2010).
[CrossRef]

Lin, Z.

Z. Lin, M. Chen, L. Wu, Y. Ma, “The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices,” in Technical Report UILU-ENG-09-2215 (UIUC, 2009).

Liu, Q.

Ma, Y.

A. Yang, S. Sastry, A. Ganesh, Y. Ma, “Fast-minimization algorithms and an application in robust face recognition: A review,” in Proceedings of IEEE Conference on Image Processing (IEEE, 2010), pp. 1849–1852.

Z. Lin, M. Chen, L. Wu, Y. Ma, “The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices,” in Technical Report UILU-ENG-09-2215 (UIUC, 2009).

Martin, M.

Matsushita, Y.

S. Han, Y. Matsushita, I. Sato, T. Okabe, Y. Sato, “Camera spectral sensitivity estimation from a single image under unknown illumination by using fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 805–812.

McNaught, A. D.

A. D. McNaught, A. Wilkinson, Compendium of Chemical Terminology (Blackwell Science, 1997).

Mirza, I. O.

Moreno-Noguer, F.

F. Moreno-Noguer, S. Nayar, P. Belhumeur, “Optimal illumination for image and video relighting,” in Proceedings of IEE European Conference on Visual Media Production (IEE, 2005), pp. 201–210.

Nayar, S.

F. Moreno-Noguer, S. Nayar, P. Belhumeur, “Optimal illumination for image and video relighting,” in Proceedings of IEE European Conference on Visual Media Production (IEE, 2005), pp. 201–210.

Nayar, S. K.

Y. Y. Schechner, S. K. Nayar, P. N. Belhumeur, “Multiplexing for optimal lighting,” IEEE Pattern Anal. Mach. Intell. 29(8), 1339–1354 (2007).
[CrossRef]

J. Park, M. Lee, M. D. Grossberg, S. K. Nayar, “Multispectral imaging using multiplexed illumination,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2007), pp. 1–8.

Oblefias, W.

Okabe, T.

S. Han, Y. Matsushita, I. Sato, T. Okabe, Y. Sato, “Camera spectral sensitivity estimation from a single image under unknown illumination by using fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 805–812.

I. Sato, T. Okabe, Y. Sato, “Bispectral photometric stereo based on fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 270–277.

Overholt, B. F.

Panjehpour, M.

Parikh, N.

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learning 3(1), 741–755 (2010).
[CrossRef]

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Separating the fluorescence and reflectance components of coral spectra,” Appl. Opt. 40(21), 3614–3621 (2001).
[CrossRef]

Park, J.

J. Park, M. Lee, M. D. Grossberg, S. K. Nayar, “Multispectral imaging using multiplexed illumination,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2007), pp. 1–8.

Peleato, B.

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learning 3(1), 741–755 (2010).
[CrossRef]

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Separating the fluorescence and reflectance components of coral spectra,” Appl. Opt. 40(21), 3614–3621 (2001).
[CrossRef]

Prather, D. W.

Ratner, N.

N. Ratner, Y. Y. Schechner, “Illumination multiplexing within fundamental limits,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2011), pp. 1–8.

Saloma, C.

Sastry, S.

A. Yang, S. Sastry, A. Ganesh, Y. Ma, “Fast-minimization algorithms and an application in robust face recognition: A review,” in Proceedings of IEEE Conference on Image Processing (IEEE, 2010), pp. 1849–1852.

Sato, I.

I. Sato, C. Zhang, “Image-based separation of reflective and fluorescent components using illumination variant and invariant color,” IEEE Trans. Pattern Anal. 35(12), 2866–2877 (2013).
[CrossRef]

A. Lam, I. Sato, “Spectral modeling and relighting of reflective-fluorescent scenes,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2013), pp. 1452–1459.

I. Sato, T. Okabe, Y. Sato, “Bispectral photometric stereo based on fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 270–277.

S. Han, Y. Matsushita, I. Sato, T. Okabe, Y. Sato, “Camera spectral sensitivity estimation from a single image under unknown illumination by using fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 805–812.

Sato, Y.

S. Han, Y. Matsushita, I. Sato, T. Okabe, Y. Sato, “Camera spectral sensitivity estimation from a single image under unknown illumination by using fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 805–812.

I. Sato, T. Okabe, Y. Sato, “Bispectral photometric stereo based on fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 270–277.

Schechner, Y.

M. Alterman, Y. Schechner, A. Weiss, “Multiplexed fluorescence unmixing,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2010), pp. 1–8.

Schechner, Y. Y.

Y. Y. Schechner, S. K. Nayar, P. N. Belhumeur, “Multiplexing for optimal lighting,” IEEE Pattern Anal. Mach. Intell. 29(8), 1339–1354 (2007).
[CrossRef]

N. Ratner, Y. Y. Schechner, “Illumination multiplexing within fundamental limits,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2011), pp. 1–8.

Schulz, T. J.

Seidel, H.-P.

M. B. Hullin, J. Hanika, B. Ajdin, H.-P. Seidel, J. Kautz, H. P. A. Lensch, “Acquisition and analysis of bis-pectral bidirectional reflectance and reradiation distribution functions,” ACM Trans. Graphics 29(4), 1–7 (2010).
[CrossRef]

Sloane, N. J. A.

M. Harwit, N. J. A. Sloane, Hadamard Transform Optics (Academic, 1979).

Soriano, M.

Springsteen, A.

A. Springsteen, “Introduction to measurement of color of fluorescent materials,” Anal. Chim. Acta 380(2), 183–192 (1999).
[CrossRef]

Stern, A.

Tanida, J.

Turk, M.

C. Chen, D. Vaquero, M. Turk, “Illumination demultiplexing from a single image,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2011), pp. 17–24.

Vaquero, D.

C. Chen, D. Vaquero, M. Turk, “Illumination demultiplexing from a single image,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2011), pp. 17–24.

Vo-Dinh, T.

Wagadarikar, A.

Weiss, A.

M. Alterman, Y. Schechner, A. Weiss, “Multiplexed fluorescence unmixing,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2010), pp. 1–8.

Wetzstein, G.

G. Wetzstein, I. Ihrke, W. Heidrich, “On plenoptic multiplexing and reconstruction,” Int. J. Comput. Vision 101(2), 384–400 (2013).
[CrossRef]

Wilkinson, A.

A. D. McNaught, A. Wilkinson, Compendium of Chemical Terminology (Blackwell Science, 1997).

Willett, R.

Willett, R. M.

Wintenberg, A.

Wu, L.

Z. Lin, M. Chen, L. Wu, Y. Ma, “The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices,” in Technical Report UILU-ENG-09-2215 (UIUC, 2009).

Wu, Y.

Xiao, X.

Yang, A.

A. Yang, S. Sastry, A. Ganesh, Y. Ma, “Fast-minimization algorithms and an application in robust face recognition: A review,” in Proceedings of IEEE Conference on Image Processing (IEEE, 2010), pp. 1849–1852.

Yoo, H.

C. Chi, H. Yoo, M. Ben-Ezra, “Multi-spectral imaging by optimized wide band illumination,” Int. J. Comput. Vision 86(2–3), 140–151 (2010).
[CrossRef]

Zhang, C.

I. Sato, C. Zhang, “Image-based separation of reflective and fluorescent components using illumination variant and invariant color,” IEEE Trans. Pattern Anal. 35(12), 2866–2877 (2013).
[CrossRef]

Zhang, Z.

Y. Deng, Q. Dai, Z. Zhang, “An overview of computational sparse models and their applications in artificial intelligence,” Artif. Intell. Evol. Comput. Metaheuristics 427, 345–369 (2012).
[CrossRef]

ACM Trans. Graphics (1)

M. B. Hullin, J. Hanika, B. Ajdin, H.-P. Seidel, J. Kautz, H. P. A. Lensch, “Acquisition and analysis of bis-pectral bidirectional reflectance and reradiation distribution functions,” ACM Trans. Graphics 29(4), 1–7 (2010).
[CrossRef]

Anal. Chim. Acta (1)

A. Springsteen, “Introduction to measurement of color of fluorescent materials,” Anal. Chim. Acta 380(2), 183–192 (1999).
[CrossRef]

Appl. Opt. (2)

Artif. Intell. Evol. Comput. Metaheuristics (1)

Y. Deng, Q. Dai, Z. Zhang, “An overview of computational sparse models and their applications in artificial intelligence,” Artif. Intell. Evol. Comput. Metaheuristics 427, 345–369 (2012).
[CrossRef]

Br. J. Appl. Phys. (1)

R. Donaldson, “Spectrophotometry of fluorescent pigments,” Br. J. Appl. Phys. 5(6), 210–214 (1954).
[CrossRef]

Found. Trends Mach. Learning (1)

S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learning 3(1), 741–755 (2010).
[CrossRef]

IEEE Comput. Graphics Appl. (1)

G. M. Johnson, M. D. Fairchild, “Full-spectral color calculations in realistic image synthesis,” IEEE Comput. Graphics Appl. 19(4), 47–53 (1999).
[CrossRef]

IEEE Pattern Anal. Mach. Intell. (1)

Y. Y. Schechner, S. K. Nayar, P. N. Belhumeur, “Multiplexing for optimal lighting,” IEEE Pattern Anal. Mach. Intell. 29(8), 1339–1354 (2007).
[CrossRef]

IEEE Trans. Pattern Anal. (1)

I. Sato, C. Zhang, “Image-based separation of reflective and fluorescent components using illumination variant and invariant color,” IEEE Trans. Pattern Anal. 35(12), 2866–2877 (2013).
[CrossRef]

Int. J. Comput. Vision (2)

C. Chi, H. Yoo, M. Ben-Ezra, “Multi-spectral imaging by optimized wide band illumination,” Int. J. Comput. Vision 86(2–3), 140–151 (2010).
[CrossRef]

G. Wetzstein, I. Ihrke, W. Heidrich, “On plenoptic multiplexing and reconstruction,” Int. J. Comput. Vision 101(2), 384–400 (2013).
[CrossRef]

Opt. Express (5)

Opt. Lett. (2)

Proc. SPIE (1)

T. Vo-Dinh, “Principle of synchronous luminescence (SL) technique for biomedical diagnostics,” Proc. SPIE 3911, 42–49 (2000).
[CrossRef]

Other (12)

I. Sato, T. Okabe, Y. Sato, “Bispectral photometric stereo based on fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 270–277.

S. Han, Y. Matsushita, I. Sato, T. Okabe, Y. Sato, “Camera spectral sensitivity estimation from a single image under unknown illumination by using fluorescence,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 805–812.

A. D. McNaught, A. Wilkinson, Compendium of Chemical Terminology (Blackwell Science, 1997).

N. Ratner, Y. Y. Schechner, “Illumination multiplexing within fundamental limits,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2011), pp. 1–8.

M. Alterman, Y. Schechner, A. Weiss, “Multiplexed fluorescence unmixing,” in Proceedings of IEEE International Conference on Computational Photography (IEEE, 2010), pp. 1–8.

J. Park, M. Lee, M. D. Grossberg, S. K. Nayar, “Multispectral imaging using multiplexed illumination,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2007), pp. 1–8.

C. Chen, D. Vaquero, M. Turk, “Illumination demultiplexing from a single image,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2011), pp. 17–24.

F. Moreno-Noguer, S. Nayar, P. Belhumeur, “Optimal illumination for image and video relighting,” in Proceedings of IEE European Conference on Visual Media Production (IEE, 2005), pp. 201–210.

A. Yang, S. Sastry, A. Ganesh, Y. Ma, “Fast-minimization algorithms and an application in robust face recognition: A review,” in Proceedings of IEEE Conference on Image Processing (IEEE, 2010), pp. 1849–1852.

Z. Lin, M. Chen, L. Wu, Y. Ma, “The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices,” in Technical Report UILU-ENG-09-2215 (UIUC, 2009).

M. Harwit, N. J. A. Sloane, Hadamard Transform Optics (Academic, 1979).

A. Lam, I. Sato, “Spectral modeling and relighting of reflective-fluorescent scenes,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2013), pp. 1452–1459.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1

The system and results of our approach on one exemplary scene. (a) Prototype setup. (b) One coded image. (c, d) Reconstructed reflectance and fluorescence, respectively.

Fig. 2
Fig. 2

The visualization of excitation–emission matrix. The strength of the matrix entries is illustrated by intensity here. Vertical and horizontal color bars respectively illustrate the excitation wavelength λin and the emission wavelength, which includes the fluorescent component λ out ref and reflective component λ out fluo. (a) No overlap between excitation and emission. (b) Slight overlap between excitation and emission.

Fig. 3
Fig. 3

Extending the reconstruction of a single pixel to image lattice by normalization. Here we use different line colors to differentiate excitation wavelengths.

Fig. 4
Fig. 4

The light path of the proposed imaging system. The corresponding real setup is shown in Fig. 1(a).

Fig. 5
Fig. 5

The performance on synthetic data, including three materials (horizontal) and three sampling rates (vertical). These results are averaged over five different random codes.

Fig. 6
Fig. 6

Performance of our algorithm on a real scene and comparison with that of traverse capturing. The left column contains only reflectance, the middle column is the fluorescence excited by a single band illumination, and the right column gives the result under a mixture-spectrum illumination. The mean absolute percentage error (MAPE) of each result is labeled in the top right corner.

Fig. 7
Fig. 7

Quantitative evaluation on real data. Here three representative points are selected, and we differentiate the excitation illuminations with different colors.

Fig. 8
Fig. 8

Relighting results under three different types of light sources and comparison with true results. (a) Noon sunlight. (b) Tungsten lamp. (c) Mercury vapor lamp.

Fig. 9
Fig. 9

Segmentation assisted by high-spectral-resolution fluorescent components. (a) A scene under daylight. (b) RGB values of five labeled regions. (c) Top three discriminative features between regions 1 and 2. (d) Segmentation of car parts. (e) Top three discriminative features among regions 3, 4, and 5. (f) Segmentation of toy ball and fluorescent paint.

Tables (1)

Tables Icon

Table 1 Algorithm 1: Reconstruct reflective and fluorescent components in a scene.

Equations (17)

Equations on this page are rendered with MathJax. Learn more.

M ^ = R ^ + F ^ + N ^ ,
( F ^ * , R ^ * , N ^ * ) = argmin F ^ * + α R ^ 1 s . t . π Ω ^ ( C ^ ) = π Ω ^ ( I ^ ( F ^ + R ^ ) O ^ + N ^ ) | N ^ μ | < 3 σ .
[ C ^ 1 C ^ 2 C ^ w ] = I ^ ( [ F ^ 1 F ^ 2 F ^ w ] + [ R ^ 1 R ^ 2 R ^ w ] ) [ O ^ O ^ O ^ ] + [ N ^ 1 N ^ 2 N ^ w ] ,
C = I ( F + R ) O + N .
( F * , R * , N * ) = argmin F * + α R 1 s . t . π Ω ( C ) = π Ω ( I ( F + R ) O + N ) | N μ | < 3 σ ,
min . S 1 * + α S 2 1 s . t . S 1 = F S 2 = R C = I F O + I R O + N + E , E ( i , j ) ( i , j ) Ω = 0 , ( N μ ) ( N μ ) 9 σ 2 + ε 2 = 0 .
Lag = S 1 * + α S 2 1 + < Y 1 , S 1 F > + β 2 S 1 F F 2 + < Y 2 , I F O + I R O + N + E C > + β 2 I F O + I R O + N + E C F 2 + < Y 3 , ( N μ ) 2 9 σ 2 + ε 2 > + β 2 ( N μ ) 2 9 σ 2 + ε 2 F 2 ,
f ( S 1 ) = S 1 * + β 2 S 1 ( F β 1 Y 1 ) F 2 + C ,
S 1 ( k + 1 ) = U s β 1 ( S temp ) V T ,
s β 1 ( x ) = { x β 1 , x > β 1 x + β 1 , x < β 1 0 , others .
f ( S 2 ) = α S 2 1 + β 2 S 2 ( R β 1 Y 0 ) F 2 .
S 2 ( k + 1 ) = s α β 1 ( R β 1 Y 0 ) .
f ( E ) E = β [ E ( C I F O I R O N β 1 Y 2 ) ] ,
E ( k + 1 ) = C I F O I R O N β 1 Y 2 .
ε ( k + 1 ) = 9 σ 2 ( N μ ) ( N μ ) β 1 Y 3 .
F ( k + 1 ) = F ( k ) γ 1 f ( F ) F R ( k + 1 ) = R ( k ) γ 2 f ( R ) R N ( k + 1 ) = N ( k ) γ 3 f ( N ) N ,
f ( F ) F = β [ F 2 + I T I F O O T ( S 1 + β 1 Y 1 ) I T ( C I R O N E β 1 Y 2 ) O T ] f ( R ) R = β [ R + I T I R O O T ( S 2 + β 1 Y 0 ) I T ( C I F O N E β 1 Y 2 ) O T ] f ( N ) N = β [ 2 ( N μ ) 3 2 ( N μ ) ( 9 σ 2 ε 2 β 1 Y 3 ) + N ( C I F O I R O E β 1 Y 2 ) ] .

Metrics