Abstract

We present an imaging framework that is able to accurately reconstruct multiple depths at individual pixels from single-photon observations. Our active imaging method models the single-photon detection statistics from multiple reflectors within a pixel, and it also exploits the fact that a multi-depth profile at each pixel can be expressed as a sparse signal. We interpret the multi-depth reconstruction problem as a sparse deconvolution problem using single-photon observations, create a convex problem through discretization and relaxation, and use a modified iterative shrinkage-thresholding algorithm to efficiently solve for the optimal multi-depth solution. We experimentally demonstrate that the proposed framework is able to accurately reconstruct the depth features of an object that is behind a partially-reflecting scatterer and 4 m away from the imager with root mean-square error of 11 cm, using only 19 signal photon detections per pixel in the presence of moderate background light. In terms of root mean-square error, this is a factor of 4.2 improvement over the conventional method of Gaussian-mixture fitting for multi-depth recovery.

© 2016 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Adaptive compressed photon counting 3D imaging based on wavelet trees and depth map sparse representation

Huidong Dai, Guohua Gu, Weiji He, Ling Ye, Tianyi Mao, and Qian Chen
Opt. Express 24(23) 26080-26096 (2016)

Underwater depth imaging using time-correlated single-photon counting

Aurora Maccarone, Aongus McCarthy, Ximing Ren, Ryan E. Warburton, Andy M. Wallace, James Moffat, Yvan Petillot, and Gerald S. Buller
Opt. Express 23(26) 33911-33926 (2015)

Kilometer-range depth imaging at 1550 nm wavelength using an InGaAs/InP single-photon avalanche diode detector

Aongus McCarthy, Ximing Ren, Adriano Della Frera, Nathan R. Gemmell, Nils J. Krichel, Carmelo Scarcella, Alessandro Ruggeri, Alberto Tosi, and Gerald S. Buller
Opt. Express 21(19) 22098-22113 (2013)

References

  • View by:
  • |
  • |
  • |

  1. K. I. Chang, K. W. Bowyer, and P. J. Flynn, “An evaluation of multimodal 2D+3D face biometrics,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 619–624 (2005).
    [Crossref] [PubMed]
  2. J. Côté, J. Widlowski, R. A. Fournier, and M. M. Verstraete, “The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial LIDAR,” Remote Sens. Environ. 113, 1067–1081 (2009).
    [Crossref]
  3. C. D. Mutto, P. Zanuttigh, and G. M. Cortelazzo, Time-of-Flight Cameras and Microsoft Kinect™ (Springer-Verlag, 2012).
    [Crossref]
  4. R. Blahut, Theory of Remote Image Formation (Cambridge University Press, 2004).
    [Crossref]
  5. J. Liang, L. Gao, P. Hai, C. Li, and L. V. Wang, “Encrypted three-dimensional dynamic imaging using snapshot time-of-flight compressed ultrafast photography,” Sci. Rep. 5, 15504 (2015).
    [Crossref] [PubMed]
  6. C. Mallet and F. Bretar, “Full-waveform topographic lidar: State-of-the-art,” ISPRS J. Photogramm. Remote Sensing 64, 1–16 (2009).
    [Crossref]
  7. F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graphics 32, 45 (2013)
    [Crossref]
  8. D. Freedman, Y. Smolin, E. Krupka, I. Leichter, and M. Schmidt, “SRA: Fast removal of general multipath for ToF sensors,” in European Conference on Computer Vision (ECCV), (Springer, 2014), pp. 234–249.
  9. H. Qiao, J. Lin, Y. Lin, M. B. Hullin, and W. Dai, “Resolving transient time profile in ToF imaging via log-sum sparse regularization,” Opt. Lett. 40, 918–921 (2015).
    [Crossref] [PubMed]
  10. S. Pellegrini, G. S. Buller, J. M. Smith, A. M. Wallace, and S. Cova, “Laser-based distance measurement using picosecond resolution time-correlated single-photon counting,” Meas. Sci. Technol. 11, 712–716 (2000).
    [Crossref]
  11. A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, “First-photon imaging,” Science 343, 58–61 (2014).
    [Crossref]
  12. D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Computational 3D and reflectivity imaging with high photon efficiency,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, New York, 2014), pp. 46–50.
  13. D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Photon-efficient computational 3D and reflectivity imaging with single-photon detectors,” IEEE Trans. Computational Imaging 1, 112–125 (2015).
    [Crossref]
  14. C. Niclass, A. Rochas, P. A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
    [Crossref]
  15. C. Niclass, M. Soga, H. Matsubara, S. Kato, and M. Kagami, “A 100-m range 10-frame/s 340 96-pixel time-of-flight depth sensor in 0.18-CMOS,” IEEE J. Solid-State Circuits 48, 559–572 (2013).
    [Crossref]
  16. S. Bellisai, D. Bronzi, F. Villa, S. Tisa, A. Tosi, and F. Zappa, “Single-photon pulsed-light indirect time-of-flight 3D ranging,” Opt. Express 21, 5086–5098 (2013).
    [Crossref] [PubMed]
  17. Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, and S. McLaughlin, “Lidar waveform based analysis of depth images constructed using sparse single-photon data,” arXiv:1507.02511 [stat:AP] (2015).
  18. G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).
  19. J. Castorena, C. D. Creusere, and D. Voelz, “Using finite moment rate of innovation for lidar waveform complexity estimation,” in Conference Record of the 44th Asilomar Conference on Signals, Systems and Computers, (IEEE, New York, 2010), pp. 608–612.
  20. J. Castorena and C. D. Creusere, “Compressive sampling of LIDAR: Full-waveforms as signals of finite rate of innovation,” in Proceedings of the 20th European Signal Processing Conference, (IEEE, New York, 2012), pp. 984–988.
  21. A. Kirmani, A. Colaço, F. N. C. Wong, and V. K. Goyal, “Exploiting sparsity in time-of-flight range acquisition using a single time-resolved sensor,” Opt. Express 19, 21485–21507 (2011).
    [Crossref] [PubMed]
  22. A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Roy. Statist. Soc., Ser. B 39, 1–38 (1977).
  23. A. Wallace, J. Ye, N. Krichel, A. McCarthy, R. J. Collins, and G. S. Buller, “Full waveform analysis for long-range 3D imaging laser radar,” EURASIP J. Adv. Signal Process. 2010, 896708 (2010).
    [Crossref]
  24. S. Hernandez-Marin, A. M. Wallace, and G. J. Gibson, “Creating multi-layered 3D images using reversible jump MCMC algorithms,” in Advances in Visual Computing, G. Bebis, R. Boyle, B. Parvin, D. Koracin, P. Remagnino, A. Nefian, G. Meenakshisundaram, V. Pascucci, J. Zara, J. Molineros, H. Theisel, and T. Malzbender., eds. (Springer, Berlin, 2006), pp. 405–416.
  25. I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Commun. Pure Appl. Math. 57, 1413–1457 (2004).
    [Crossref]
  26. A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imag. Sci. 2, 183–202 (2009).
    [Crossref]
  27. D. Snyder, Random Point Processes (Wiley, 1975).
  28. B. K. Natarajan, “Sparse approximate solutions to linear systems,” SIAM J. Comput. 24, 227–234 (1995).
    [Crossref]
  29. M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing (Springer, 2010).
    [Crossref]
  30. S. Fuchs, M. Suppa, and O. Hellwich, “Compensation for multipath in ToF camera measurements supported by photometric calibration and environment integration,” in Computer Vision Systems, M. Chen, B. Leibe, and B. Neumann, eds. (Springer, 2013), pp. 31–41.
    [Crossref]
  31. A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
    [Crossref] [PubMed]
  32. “GitHub repository for multi-depth single-photon imaging,” https://github.com/photon-efficient-imaging/full-waveform/ .
  33. D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
    [Crossref]
  34. G. H. Golub, M. Heath, and G. Wahba, “Generalized cross-validation as a method for choosing a good ridge parameter,” Technometrics 21, 215–223 (1979).
    [Crossref]

2015 (4)

J. Liang, L. Gao, P. Hai, C. Li, and L. V. Wang, “Encrypted three-dimensional dynamic imaging using snapshot time-of-flight compressed ultrafast photography,” Sci. Rep. 5, 15504 (2015).
[Crossref] [PubMed]

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Photon-efficient computational 3D and reflectivity imaging with single-photon detectors,” IEEE Trans. Computational Imaging 1, 112–125 (2015).
[Crossref]

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

H. Qiao, J. Lin, Y. Lin, M. B. Hullin, and W. Dai, “Resolving transient time profile in ToF imaging via log-sum sparse regularization,” Opt. Lett. 40, 918–921 (2015).
[Crossref] [PubMed]

2014 (2)

2013 (3)

S. Bellisai, D. Bronzi, F. Villa, S. Tisa, A. Tosi, and F. Zappa, “Single-photon pulsed-light indirect time-of-flight 3D ranging,” Opt. Express 21, 5086–5098 (2013).
[Crossref] [PubMed]

C. Niclass, M. Soga, H. Matsubara, S. Kato, and M. Kagami, “A 100-m range 10-frame/s 340 96-pixel time-of-flight depth sensor in 0.18-CMOS,” IEEE J. Solid-State Circuits 48, 559–572 (2013).
[Crossref]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graphics 32, 45 (2013)
[Crossref]

2011 (1)

2010 (1)

A. Wallace, J. Ye, N. Krichel, A. McCarthy, R. J. Collins, and G. S. Buller, “Full waveform analysis for long-range 3D imaging laser radar,” EURASIP J. Adv. Signal Process. 2010, 896708 (2010).
[Crossref]

2009 (3)

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imag. Sci. 2, 183–202 (2009).
[Crossref]

C. Mallet and F. Bretar, “Full-waveform topographic lidar: State-of-the-art,” ISPRS J. Photogramm. Remote Sensing 64, 1–16 (2009).
[Crossref]

J. Côté, J. Widlowski, R. A. Fournier, and M. M. Verstraete, “The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial LIDAR,” Remote Sens. Environ. 113, 1067–1081 (2009).
[Crossref]

2005 (2)

K. I. Chang, K. W. Bowyer, and P. J. Flynn, “An evaluation of multimodal 2D+3D face biometrics,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 619–624 (2005).
[Crossref] [PubMed]

C. Niclass, A. Rochas, P. A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

2004 (1)

I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Commun. Pure Appl. Math. 57, 1413–1457 (2004).
[Crossref]

2000 (1)

S. Pellegrini, G. S. Buller, J. M. Smith, A. M. Wallace, and S. Cova, “Laser-based distance measurement using picosecond resolution time-correlated single-photon counting,” Meas. Sci. Technol. 11, 712–716 (2000).
[Crossref]

1995 (1)

B. K. Natarajan, “Sparse approximate solutions to linear systems,” SIAM J. Comput. 24, 227–234 (1995).
[Crossref]

1979 (1)

G. H. Golub, M. Heath, and G. Wahba, “Generalized cross-validation as a method for choosing a good ridge parameter,” Technometrics 21, 215–223 (1979).
[Crossref]

1977 (1)

A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Roy. Statist. Soc., Ser. B 39, 1–38 (1977).

Altmann, Y.

Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, and S. McLaughlin, “Lidar waveform based analysis of depth images constructed using sparse single-photon data,” arXiv:1507.02511 [stat:AP] (2015).

Barsi, C.

Beck, A.

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imag. Sci. 2, 183–202 (2009).
[Crossref]

Bellisai, S.

Besse, P. A.

C. Niclass, A. Rochas, P. A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

Bhandari, A.

Blahut, R.

R. Blahut, Theory of Remote Image Formation (Cambridge University Press, 2004).
[Crossref]

Bowyer, K. W.

K. I. Chang, K. W. Bowyer, and P. J. Flynn, “An evaluation of multimodal 2D+3D face biometrics,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 619–624 (2005).
[Crossref] [PubMed]

Bretar, F.

C. Mallet and F. Bretar, “Full-waveform topographic lidar: State-of-the-art,” ISPRS J. Photogramm. Remote Sensing 64, 1–16 (2009).
[Crossref]

Bronzi, D.

Buller, G. S.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

A. Wallace, J. Ye, N. Krichel, A. McCarthy, R. J. Collins, and G. S. Buller, “Full waveform analysis for long-range 3D imaging laser radar,” EURASIP J. Adv. Signal Process. 2010, 896708 (2010).
[Crossref]

S. Pellegrini, G. S. Buller, J. M. Smith, A. M. Wallace, and S. Cova, “Laser-based distance measurement using picosecond resolution time-correlated single-photon counting,” Meas. Sci. Technol. 11, 712–716 (2000).
[Crossref]

Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, and S. McLaughlin, “Lidar waveform based analysis of depth images constructed using sparse single-photon data,” arXiv:1507.02511 [stat:AP] (2015).

Castorena, J.

J. Castorena and C. D. Creusere, “Compressive sampling of LIDAR: Full-waveforms as signals of finite rate of innovation,” in Proceedings of the 20th European Signal Processing Conference, (IEEE, New York, 2012), pp. 984–988.

J. Castorena, C. D. Creusere, and D. Voelz, “Using finite moment rate of innovation for lidar waveform complexity estimation,” in Conference Record of the 44th Asilomar Conference on Signals, Systems and Computers, (IEEE, New York, 2010), pp. 608–612.

Chang, K. I.

K. I. Chang, K. W. Bowyer, and P. J. Flynn, “An evaluation of multimodal 2D+3D face biometrics,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 619–624 (2005).
[Crossref] [PubMed]

Charbon, E.

C. Niclass, A. Rochas, P. A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

Colaço, A.

A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, “First-photon imaging,” Science 343, 58–61 (2014).
[Crossref]

A. Kirmani, A. Colaço, F. N. C. Wong, and V. K. Goyal, “Exploiting sparsity in time-of-flight range acquisition using a single time-resolved sensor,” Opt. Express 19, 21485–21507 (2011).
[Crossref] [PubMed]

Collins, R. J.

A. Wallace, J. Ye, N. Krichel, A. McCarthy, R. J. Collins, and G. S. Buller, “Full waveform analysis for long-range 3D imaging laser radar,” EURASIP J. Adv. Signal Process. 2010, 896708 (2010).
[Crossref]

Cortelazzo, G. M.

C. D. Mutto, P. Zanuttigh, and G. M. Cortelazzo, Time-of-Flight Cameras and Microsoft Kinect™ (Springer-Verlag, 2012).
[Crossref]

Côté, J.

J. Côté, J. Widlowski, R. A. Fournier, and M. M. Verstraete, “The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial LIDAR,” Remote Sens. Environ. 113, 1067–1081 (2009).
[Crossref]

Cova, S.

S. Pellegrini, G. S. Buller, J. M. Smith, A. M. Wallace, and S. Cova, “Laser-based distance measurement using picosecond resolution time-correlated single-photon counting,” Meas. Sci. Technol. 11, 712–716 (2000).
[Crossref]

Cree, M.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
[Crossref]

Creusere, C. D.

J. Castorena, C. D. Creusere, and D. Voelz, “Using finite moment rate of innovation for lidar waveform complexity estimation,” in Conference Record of the 44th Asilomar Conference on Signals, Systems and Computers, (IEEE, New York, 2010), pp. 608–612.

J. Castorena and C. D. Creusere, “Compressive sampling of LIDAR: Full-waveforms as signals of finite rate of innovation,” in Proceedings of the 20th European Signal Processing Conference, (IEEE, New York, 2012), pp. 984–988.

Dai, W.

Daubechies, I.

I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Commun. Pure Appl. Math. 57, 1413–1457 (2004).
[Crossref]

De Mol, C.

I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Commun. Pure Appl. Math. 57, 1413–1457 (2004).
[Crossref]

Defrise, M.

I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Commun. Pure Appl. Math. 57, 1413–1457 (2004).
[Crossref]

Dempster, A. P.

A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Roy. Statist. Soc., Ser. B 39, 1–38 (1977).

Dorrington, A.

Elad, M.

M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing (Springer, 2010).
[Crossref]

Faccio, D.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

Feigin, M.

Flynn, P. J.

K. I. Chang, K. W. Bowyer, and P. J. Flynn, “An evaluation of multimodal 2D+3D face biometrics,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 619–624 (2005).
[Crossref] [PubMed]

Fournier, R. A.

J. Côté, J. Widlowski, R. A. Fournier, and M. M. Verstraete, “The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial LIDAR,” Remote Sens. Environ. 113, 1067–1081 (2009).
[Crossref]

Freedman, D.

D. Freedman, Y. Smolin, E. Krupka, I. Leichter, and M. Schmidt, “SRA: Fast removal of general multipath for ToF sensors,” in European Conference on Computer Vision (ECCV), (Springer, 2014), pp. 234–249.

Fuchs, S.

S. Fuchs, M. Suppa, and O. Hellwich, “Compensation for multipath in ToF camera measurements supported by photometric calibration and environment integration,” in Computer Vision Systems, M. Chen, B. Leibe, and B. Neumann, eds. (Springer, 2013), pp. 31–41.
[Crossref]

Gao, L.

J. Liang, L. Gao, P. Hai, C. Li, and L. V. Wang, “Encrypted three-dimensional dynamic imaging using snapshot time-of-flight compressed ultrafast photography,” Sci. Rep. 5, 15504 (2015).
[Crossref] [PubMed]

Gariepy, G.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

Gibson, G. J.

S. Hernandez-Marin, A. M. Wallace, and G. J. Gibson, “Creating multi-layered 3D images using reversible jump MCMC algorithms,” in Advances in Visual Computing, G. Bebis, R. Boyle, B. Parvin, D. Koracin, P. Remagnino, A. Nefian, G. Meenakshisundaram, V. Pascucci, J. Zara, J. Molineros, H. Theisel, and T. Malzbender., eds. (Springer, Berlin, 2006), pp. 405–416.

Golub, G. H.

G. H. Golub, M. Heath, and G. Wahba, “Generalized cross-validation as a method for choosing a good ridge parameter,” Technometrics 21, 215–223 (1979).
[Crossref]

Goyal, V. K.

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Photon-efficient computational 3D and reflectivity imaging with single-photon detectors,” IEEE Trans. Computational Imaging 1, 112–125 (2015).
[Crossref]

A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, “First-photon imaging,” Science 343, 58–61 (2014).
[Crossref]

A. Kirmani, A. Colaço, F. N. C. Wong, and V. K. Goyal, “Exploiting sparsity in time-of-flight range acquisition using a single time-resolved sensor,” Opt. Express 19, 21485–21507 (2011).
[Crossref] [PubMed]

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Computational 3D and reflectivity imaging with high photon efficiency,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, New York, 2014), pp. 46–50.

Gregson, J.

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graphics 32, 45 (2013)
[Crossref]

Hai, P.

J. Liang, L. Gao, P. Hai, C. Li, and L. V. Wang, “Encrypted three-dimensional dynamic imaging using snapshot time-of-flight compressed ultrafast photography,” Sci. Rep. 5, 15504 (2015).
[Crossref] [PubMed]

Heath, M.

G. H. Golub, M. Heath, and G. Wahba, “Generalized cross-validation as a method for choosing a good ridge parameter,” Technometrics 21, 215–223 (1979).
[Crossref]

Heide, F.

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graphics 32, 45 (2013)
[Crossref]

Heidrich, W.

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graphics 32, 45 (2013)
[Crossref]

Hellwich, O.

S. Fuchs, M. Suppa, and O. Hellwich, “Compensation for multipath in ToF camera measurements supported by photometric calibration and environment integration,” in Computer Vision Systems, M. Chen, B. Leibe, and B. Neumann, eds. (Springer, 2013), pp. 31–41.
[Crossref]

Henderson, R.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

Hernandez-Marin, S.

S. Hernandez-Marin, A. M. Wallace, and G. J. Gibson, “Creating multi-layered 3D images using reversible jump MCMC algorithms,” in Advances in Visual Computing, G. Bebis, R. Boyle, B. Parvin, D. Koracin, P. Remagnino, A. Nefian, G. Meenakshisundaram, V. Pascucci, J. Zara, J. Molineros, H. Theisel, and T. Malzbender., eds. (Springer, Berlin, 2006), pp. 405–416.

Heshmat, B.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

Hullin, M. B.

H. Qiao, J. Lin, Y. Lin, M. B. Hullin, and W. Dai, “Resolving transient time profile in ToF imaging via log-sum sparse regularization,” Opt. Lett. 40, 918–921 (2015).
[Crossref] [PubMed]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graphics 32, 45 (2013)
[Crossref]

Kadambi, A.

Kagami, M.

C. Niclass, M. Soga, H. Matsubara, S. Kato, and M. Kagami, “A 100-m range 10-frame/s 340 96-pixel time-of-flight depth sensor in 0.18-CMOS,” IEEE J. Solid-State Circuits 48, 559–572 (2013).
[Crossref]

Kato, S.

C. Niclass, M. Soga, H. Matsubara, S. Kato, and M. Kagami, “A 100-m range 10-frame/s 340 96-pixel time-of-flight depth sensor in 0.18-CMOS,” IEEE J. Solid-State Circuits 48, 559–572 (2013).
[Crossref]

Kirmani, A.

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Photon-efficient computational 3D and reflectivity imaging with single-photon detectors,” IEEE Trans. Computational Imaging 1, 112–125 (2015).
[Crossref]

A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, “First-photon imaging,” Science 343, 58–61 (2014).
[Crossref]

A. Kirmani, A. Colaço, F. N. C. Wong, and V. K. Goyal, “Exploiting sparsity in time-of-flight range acquisition using a single time-resolved sensor,” Opt. Express 19, 21485–21507 (2011).
[Crossref] [PubMed]

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Computational 3D and reflectivity imaging with high photon efficiency,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, New York, 2014), pp. 46–50.

Koch, R.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
[Crossref]

Kolb, A.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
[Crossref]

Krichel, N.

A. Wallace, J. Ye, N. Krichel, A. McCarthy, R. J. Collins, and G. S. Buller, “Full waveform analysis for long-range 3D imaging laser radar,” EURASIP J. Adv. Signal Process. 2010, 896708 (2010).
[Crossref]

Krstajic, N.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

Krupka, E.

D. Freedman, Y. Smolin, E. Krupka, I. Leichter, and M. Schmidt, “SRA: Fast removal of general multipath for ToF sensors,” in European Conference on Computer Vision (ECCV), (Springer, 2014), pp. 234–249.

Laird, N. M.

A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Roy. Statist. Soc., Ser. B 39, 1–38 (1977).

Leach, J.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

Lefloch, D.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
[Crossref]

Leichter, I.

D. Freedman, Y. Smolin, E. Krupka, I. Leichter, and M. Schmidt, “SRA: Fast removal of general multipath for ToF sensors,” in European Conference on Computer Vision (ECCV), (Springer, 2014), pp. 234–249.

Lenzen, F.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
[Crossref]

Li, C.

J. Liang, L. Gao, P. Hai, C. Li, and L. V. Wang, “Encrypted three-dimensional dynamic imaging using snapshot time-of-flight compressed ultrafast photography,” Sci. Rep. 5, 15504 (2015).
[Crossref] [PubMed]

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

Liang, J.

J. Liang, L. Gao, P. Hai, C. Li, and L. V. Wang, “Encrypted three-dimensional dynamic imaging using snapshot time-of-flight compressed ultrafast photography,” Sci. Rep. 5, 15504 (2015).
[Crossref] [PubMed]

Lin, J.

Lin, Y.

Mallet, C.

C. Mallet and F. Bretar, “Full-waveform topographic lidar: State-of-the-art,” ISPRS J. Photogramm. Remote Sensing 64, 1–16 (2009).
[Crossref]

Matsubara, H.

C. Niclass, M. Soga, H. Matsubara, S. Kato, and M. Kagami, “A 100-m range 10-frame/s 340 96-pixel time-of-flight depth sensor in 0.18-CMOS,” IEEE J. Solid-State Circuits 48, 559–572 (2013).
[Crossref]

McCarthy, A.

A. Wallace, J. Ye, N. Krichel, A. McCarthy, R. J. Collins, and G. S. Buller, “Full waveform analysis for long-range 3D imaging laser radar,” EURASIP J. Adv. Signal Process. 2010, 896708 (2010).
[Crossref]

Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, and S. McLaughlin, “Lidar waveform based analysis of depth images constructed using sparse single-photon data,” arXiv:1507.02511 [stat:AP] (2015).

McLaughlin, S.

Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, and S. McLaughlin, “Lidar waveform based analysis of depth images constructed using sparse single-photon data,” arXiv:1507.02511 [stat:AP] (2015).

Mutto, C. D.

C. D. Mutto, P. Zanuttigh, and G. M. Cortelazzo, Time-of-Flight Cameras and Microsoft Kinect™ (Springer-Verlag, 2012).
[Crossref]

Nair, R.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
[Crossref]

Natarajan, B. K.

B. K. Natarajan, “Sparse approximate solutions to linear systems,” SIAM J. Comput. 24, 227–234 (1995).
[Crossref]

Niclass, C.

C. Niclass, M. Soga, H. Matsubara, S. Kato, and M. Kagami, “A 100-m range 10-frame/s 340 96-pixel time-of-flight depth sensor in 0.18-CMOS,” IEEE J. Solid-State Circuits 48, 559–572 (2013).
[Crossref]

C. Niclass, A. Rochas, P. A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

Pellegrini, S.

S. Pellegrini, G. S. Buller, J. M. Smith, A. M. Wallace, and S. Cova, “Laser-based distance measurement using picosecond resolution time-correlated single-photon counting,” Meas. Sci. Technol. 11, 712–716 (2000).
[Crossref]

Qiao, H.

Raskar, R.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

A. Bhandari, A. Kadambi, R. Whyte, C. Barsi, M. Feigin, A. Dorrington, and R. Raskar, “Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization,” Opt. Lett. 39, 1705–1708 (2014).
[Crossref] [PubMed]

Ren, X.

Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, and S. McLaughlin, “Lidar waveform based analysis of depth images constructed using sparse single-photon data,” arXiv:1507.02511 [stat:AP] (2015).

Rochas, A.

C. Niclass, A. Rochas, P. A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

Rubin, D. B.

A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Roy. Statist. Soc., Ser. B 39, 1–38 (1977).

Schäfer, H.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
[Crossref]

Schmidt, M.

D. Freedman, Y. Smolin, E. Krupka, I. Leichter, and M. Schmidt, “SRA: Fast removal of general multipath for ToF sensors,” in European Conference on Computer Vision (ECCV), (Springer, 2014), pp. 234–249.

Shapiro, J. H.

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Photon-efficient computational 3D and reflectivity imaging with single-photon detectors,” IEEE Trans. Computational Imaging 1, 112–125 (2015).
[Crossref]

A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, “First-photon imaging,” Science 343, 58–61 (2014).
[Crossref]

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Computational 3D and reflectivity imaging with high photon efficiency,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, New York, 2014), pp. 46–50.

Shin, D.

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Photon-efficient computational 3D and reflectivity imaging with single-photon detectors,” IEEE Trans. Computational Imaging 1, 112–125 (2015).
[Crossref]

A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, “First-photon imaging,” Science 343, 58–61 (2014).
[Crossref]

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Computational 3D and reflectivity imaging with high photon efficiency,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, New York, 2014), pp. 46–50.

Smith, J. M.

S. Pellegrini, G. S. Buller, J. M. Smith, A. M. Wallace, and S. Cova, “Laser-based distance measurement using picosecond resolution time-correlated single-photon counting,” Meas. Sci. Technol. 11, 712–716 (2000).
[Crossref]

Smolin, Y.

D. Freedman, Y. Smolin, E. Krupka, I. Leichter, and M. Schmidt, “SRA: Fast removal of general multipath for ToF sensors,” in European Conference on Computer Vision (ECCV), (Springer, 2014), pp. 234–249.

Snyder, D.

D. Snyder, Random Point Processes (Wiley, 1975).

Soga, M.

C. Niclass, M. Soga, H. Matsubara, S. Kato, and M. Kagami, “A 100-m range 10-frame/s 340 96-pixel time-of-flight depth sensor in 0.18-CMOS,” IEEE J. Solid-State Circuits 48, 559–572 (2013).
[Crossref]

Streeter, L.

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
[Crossref]

Suppa, M.

S. Fuchs, M. Suppa, and O. Hellwich, “Compensation for multipath in ToF camera measurements supported by photometric calibration and environment integration,” in Computer Vision Systems, M. Chen, B. Leibe, and B. Neumann, eds. (Springer, 2013), pp. 31–41.
[Crossref]

Teboulle, M.

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imag. Sci. 2, 183–202 (2009).
[Crossref]

Thomson, R. R.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

Tisa, S.

Tosi, A.

Venkatraman, D.

A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, “First-photon imaging,” Science 343, 58–61 (2014).
[Crossref]

Verstraete, M. M.

J. Côté, J. Widlowski, R. A. Fournier, and M. M. Verstraete, “The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial LIDAR,” Remote Sens. Environ. 113, 1067–1081 (2009).
[Crossref]

Villa, F.

Voelz, D.

J. Castorena, C. D. Creusere, and D. Voelz, “Using finite moment rate of innovation for lidar waveform complexity estimation,” in Conference Record of the 44th Asilomar Conference on Signals, Systems and Computers, (IEEE, New York, 2010), pp. 608–612.

Wahba, G.

G. H. Golub, M. Heath, and G. Wahba, “Generalized cross-validation as a method for choosing a good ridge parameter,” Technometrics 21, 215–223 (1979).
[Crossref]

Wallace, A.

A. Wallace, J. Ye, N. Krichel, A. McCarthy, R. J. Collins, and G. S. Buller, “Full waveform analysis for long-range 3D imaging laser radar,” EURASIP J. Adv. Signal Process. 2010, 896708 (2010).
[Crossref]

Wallace, A. M.

S. Pellegrini, G. S. Buller, J. M. Smith, A. M. Wallace, and S. Cova, “Laser-based distance measurement using picosecond resolution time-correlated single-photon counting,” Meas. Sci. Technol. 11, 712–716 (2000).
[Crossref]

S. Hernandez-Marin, A. M. Wallace, and G. J. Gibson, “Creating multi-layered 3D images using reversible jump MCMC algorithms,” in Advances in Visual Computing, G. Bebis, R. Boyle, B. Parvin, D. Koracin, P. Remagnino, A. Nefian, G. Meenakshisundaram, V. Pascucci, J. Zara, J. Molineros, H. Theisel, and T. Malzbender., eds. (Springer, Berlin, 2006), pp. 405–416.

Wang, L. V.

J. Liang, L. Gao, P. Hai, C. Li, and L. V. Wang, “Encrypted three-dimensional dynamic imaging using snapshot time-of-flight compressed ultrafast photography,” Sci. Rep. 5, 15504 (2015).
[Crossref] [PubMed]

Whyte, R.

Widlowski, J.

J. Côté, J. Widlowski, R. A. Fournier, and M. M. Verstraete, “The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial LIDAR,” Remote Sens. Environ. 113, 1067–1081 (2009).
[Crossref]

Wong, F. N. C.

A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, “First-photon imaging,” Science 343, 58–61 (2014).
[Crossref]

A. Kirmani, A. Colaço, F. N. C. Wong, and V. K. Goyal, “Exploiting sparsity in time-of-flight range acquisition using a single time-resolved sensor,” Opt. Express 19, 21485–21507 (2011).
[Crossref] [PubMed]

Ye, J.

A. Wallace, J. Ye, N. Krichel, A. McCarthy, R. J. Collins, and G. S. Buller, “Full waveform analysis for long-range 3D imaging laser radar,” EURASIP J. Adv. Signal Process. 2010, 896708 (2010).
[Crossref]

Zanuttigh, P.

C. D. Mutto, P. Zanuttigh, and G. M. Cortelazzo, Time-of-Flight Cameras and Microsoft Kinect™ (Springer-Verlag, 2012).
[Crossref]

Zappa, F.

ACM Trans. Graphics (1)

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graphics 32, 45 (2013)
[Crossref]

Commun. Pure Appl. Math. (1)

I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Commun. Pure Appl. Math. 57, 1413–1457 (2004).
[Crossref]

EURASIP J. Adv. Signal Process. (1)

A. Wallace, J. Ye, N. Krichel, A. McCarthy, R. J. Collins, and G. S. Buller, “Full waveform analysis for long-range 3D imaging laser radar,” EURASIP J. Adv. Signal Process. 2010, 896708 (2010).
[Crossref]

IEEE J. Solid-State Circuits (2)

C. Niclass, A. Rochas, P. A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

C. Niclass, M. Soga, H. Matsubara, S. Kato, and M. Kagami, “A 100-m range 10-frame/s 340 96-pixel time-of-flight depth sensor in 0.18-CMOS,” IEEE J. Solid-State Circuits 48, 559–572 (2013).
[Crossref]

IEEE Trans. Computational Imaging (1)

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Photon-efficient computational 3D and reflectivity imaging with single-photon detectors,” IEEE Trans. Computational Imaging 1, 112–125 (2015).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

K. I. Chang, K. W. Bowyer, and P. J. Flynn, “An evaluation of multimodal 2D+3D face biometrics,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 619–624 (2005).
[Crossref] [PubMed]

ISPRS J. Photogramm. Remote Sensing (1)

C. Mallet and F. Bretar, “Full-waveform topographic lidar: State-of-the-art,” ISPRS J. Photogramm. Remote Sensing 64, 1–16 (2009).
[Crossref]

J. Roy. Statist. Soc., Ser. B (1)

A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Roy. Statist. Soc., Ser. B 39, 1–38 (1977).

Meas. Sci. Technol. (1)

S. Pellegrini, G. S. Buller, J. M. Smith, A. M. Wallace, and S. Cova, “Laser-based distance measurement using picosecond resolution time-correlated single-photon counting,” Meas. Sci. Technol. 11, 712–716 (2000).
[Crossref]

Nat. Commun. (1)

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nat. Commun. 6, 7021 (2015).

Opt. Express (2)

Opt. Lett. (2)

Remote Sens. Environ. (1)

J. Côté, J. Widlowski, R. A. Fournier, and M. M. Verstraete, “The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial LIDAR,” Remote Sens. Environ. 113, 1067–1081 (2009).
[Crossref]

Sci. Rep. (1)

J. Liang, L. Gao, P. Hai, C. Li, and L. V. Wang, “Encrypted three-dimensional dynamic imaging using snapshot time-of-flight compressed ultrafast photography,” Sci. Rep. 5, 15504 (2015).
[Crossref] [PubMed]

Science (1)

A. Kirmani, D. Venkatraman, D. Shin, A. Colaço, F. N. C. Wong, J. H. Shapiro, and V. K. Goyal, “First-photon imaging,” Science 343, 58–61 (2014).
[Crossref]

SIAM J. Comput. (1)

B. K. Natarajan, “Sparse approximate solutions to linear systems,” SIAM J. Comput. 24, 227–234 (1995).
[Crossref]

SIAM J. Imag. Sci. (1)

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imag. Sci. 2, 183–202 (2009).
[Crossref]

Technometrics (1)

G. H. Golub, M. Heath, and G. Wahba, “Generalized cross-validation as a method for choosing a good ridge parameter,” Technometrics 21, 215–223 (1979).
[Crossref]

Other (13)

D. Snyder, Random Point Processes (Wiley, 1975).

S. Hernandez-Marin, A. M. Wallace, and G. J. Gibson, “Creating multi-layered 3D images using reversible jump MCMC algorithms,” in Advances in Visual Computing, G. Bebis, R. Boyle, B. Parvin, D. Koracin, P. Remagnino, A. Nefian, G. Meenakshisundaram, V. Pascucci, J. Zara, J. Molineros, H. Theisel, and T. Malzbender., eds. (Springer, Berlin, 2006), pp. 405–416.

M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing (Springer, 2010).
[Crossref]

S. Fuchs, M. Suppa, and O. Hellwich, “Compensation for multipath in ToF camera measurements supported by photometric calibration and environment integration,” in Computer Vision Systems, M. Chen, B. Leibe, and B. Neumann, eds. (Springer, 2013), pp. 31–41.
[Crossref]

“GitHub repository for multi-depth single-photon imaging,” https://github.com/photon-efficient-imaging/full-waveform/ .

D. Lefloch, R. Nair, F. Lenzen, H. Schäfer, L. Streeter, M. Cree, R. Koch, and A. Kolb, “Technical foundation and calibration methods for time-of-flight cameras,” in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, M. Grzegorzek, C. Theobalt, R. Koch, and A. Kolb, eds. (Springer, 2013), pp. 3–24.
[Crossref]

D. Shin, A. Kirmani, V. K. Goyal, and J. H. Shapiro, “Computational 3D and reflectivity imaging with high photon efficiency,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, New York, 2014), pp. 46–50.

Y. Altmann, X. Ren, A. McCarthy, G. S. Buller, and S. McLaughlin, “Lidar waveform based analysis of depth images constructed using sparse single-photon data,” arXiv:1507.02511 [stat:AP] (2015).

J. Castorena, C. D. Creusere, and D. Voelz, “Using finite moment rate of innovation for lidar waveform complexity estimation,” in Conference Record of the 44th Asilomar Conference on Signals, Systems and Computers, (IEEE, New York, 2010), pp. 608–612.

J. Castorena and C. D. Creusere, “Compressive sampling of LIDAR: Full-waveforms as signals of finite rate of innovation,” in Proceedings of the 20th European Signal Processing Conference, (IEEE, New York, 2012), pp. 984–988.

D. Freedman, Y. Smolin, E. Krupka, I. Leichter, and M. Schmidt, “SRA: Fast removal of general multipath for ToF sensors,” in European Conference on Computer Vision (ECCV), (Springer, 2014), pp. 234–249.

C. D. Mutto, P. Zanuttigh, and G. M. Cortelazzo, Time-of-Flight Cameras and Microsoft Kinect™ (Springer-Verlag, 2012).
[Crossref]

R. Blahut, Theory of Remote Image Formation (Cambridge University Press, 2004).
[Crossref]

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1

Examples of active imaging scenarios in which the scene response is a sum of responses from multiple reflectors. (a) Imaging scene with a partially-reflecting object (shown in gray dashed line). (b) Imaging scene with a partially-occluding object.

Fig. 2
Fig. 2

(Top) Full-waveform single-photon imaging setup for estimation of depths of multiple objects. In this example, a pulsed optical source illuminates a pixel of the scene that includes a partially-reflective object occluding a target of interest. The optical flux incident at the single-photon detector combines the backreflected waveform from multiple reflectors in the scene pixel with extraneous background light. (Bottom left) The photon detections, shown as spikes, are generated by the N-pulse rate function (t) following an inhomogeneous Poisson process. The green and blue spikes represent photon detections from the first and second reflector, respectively; the red spikes represent the unwanted photon detections from background light and dark counts. (Bottom right) Our convex optimization processing enables accurate reconstruction of multiple depths of reflectors in the scene from a small number of photon detections.

Fig. 3
Fig. 3

Illustration of a shrinkage-thresholding operation used as a step in ISTA (left) and the shrinkage-rectification operation used as a step in our SPISTA (right) that includes the nonnegativity constraint. Here the operations map scalar v to scalar z (variables only used for illustration purposes), with regularization parameter τ.

Fig. 4
Fig. 4

Illustration of steps of Algorithm 2 using experimental photon-count data for the single-pixel multi-depth example of partially-occluding reflectors in Section 4.2.2. (a) The raw photon count vector y obtained by imaging a pixel with two depths. Other than the photon detections describing the two targets of interest, we observe extraneous photon detections from background and dark counts. (b) The output solution of SPISTA in Algorithm 1. Note that the extraneous background and detector dark counts are suppressed. (c) The final solution of Algorithm 2 that groups depths of SPISTA output.

Fig. 5
Fig. 5

Simulated performance of MoG-based method and proposed framework in recovering signals with K = 2 for two different background levels. Signal photon detections are detections originating from scene response and do not include the background-light plus dark-count detections. Note that the units of NRMSE are in meters, after being normalized by the pulsewidth; 1 NRMSE equals cTp/2 = 4.5 cm. The plots also include error bars indicating the ±1 standard errors.

Fig. 6
Fig. 6

Simulated results of mean estimates of the number of reflectors produced by Algorithm 2 at a pixel, when the RMS pulsewidth is set to be cTp = 2 cm. Here we show plots when the reflectivity ratio between the first and second target is (a) 1 (blue line), (b) 1/2 (cyan line), (c) 1/4 (yellow line), and (d) 1/8 (red line),

Fig. 7
Fig. 7

Experimental setup with a raster-scanning source and a single SPAD detector. The red arrows show the path of the optical signal from laser source, and the green arrows show the path of the electrical signal that indicates whether a photon has been detected or not.

Fig. 8
Fig. 8

(Left) Photograph of the mannequin placed behind a partially-scattering object from the single-photon imagers’ point of view. (Right) Experimental results for estimating depth of the mannequin through the partially-reflective material using MoG-based and our estimators, given that our imaging setup is at longitudinal position z = 0. Our multi-depth results were generated using the parameters τ = 0.1, δ = 10−4, ε = 0.1, and x ^ ( 0 ) S T y.

Fig. 9
Fig. 9

Single-photon imaging setup for estimating multi-depth from partial occlusions at depth boundary pixels. Sample data from 38 photon detections is shown below for the pixel (94,230) where partial occlusions occur. We show experimental results of multi-depth recovery for this scene using the MoG-based and our methods.

Fig. 10
Fig. 10

Experimental results of depth reconstruction of sunflower occluding a wall, given that our imaging setup is at z = 0. Using our imaging framework, the mixed-pixel artifacts at the depth boundary of the flower and background light plus dark count noise are suppressed. Our multi-depth results were generated using the parameters τ = 0.2, δ = 10−4, ε = 0.1, and x ^ ( 0 ) = S T y.

Tables (2)

Tables Icon

Algorithm 1 Single-photon iterative shrinkage-thresholding algorithm (SPISTA)

Tables Icon

Algorithm 2 Computational multi-depth single-photon imaging

Equations (16)

Equations on this page are rendered with MathJax. Learn more.

r ( t ) = ( h s ) ( t ) + B ,
λ ( t ) = η ( h d r ) ( t ) + d ,
λ ( t ) = η ( h s ˜ ) ( t ) + ( η B + d ) ,
y k Poisson ( N ( k 1 ) Δ k Δ λ ( t ) d t ) = Poisson ( N η ( k 1 ) Δ k Δ ( h s ˜ ) ( t ) d t Mean count of signal photons + N Δ ( η B + d ) Mean count of background photons plus dark counts ) .
N η ( k 1 ) Δ k Δ ( h s ˜ ) ( t ) d t = ( k 1 ) Δ k Δ 0 T r N η h ( y ) s ˜ ( t y ) d y d t = ( a ) ( k 1 ) Δ k Δ j = 1 n ( j 1 ) Δ j Δ N η h ( y ) s ˜ ( t y ) d y d t = j = 1 n ( k 1 ) Δ k Δ ( j 1 ) Δ j Δ h ( y ) N η s ˜ ( t y ) d y d t ( b ) j = 1 n ( k 1 ) Δ k Δ ( j 1 ) Δ j Δ x j Δ S k , j Δ d y d t = j = 1 n S k , j x j ,
x j = ( j 1 ) Δ j Δ h ( y ) d y , for j = 1 , , n ,
S k , j = 1 Δ ( k 1 ) Δ k Δ ( j 1 ) Δ j Δ N η s ˜ | ( t y ) d y d t , for k = 1 , , m , j = 1 , , n .
y k Poisson ( ( Sx + b 1 m ) k ) , for k = 1 , 2 , , m ,
b = N Δ ( η B + d ) .
p Y ( y ; x , S , b ) = k = 1 m exp { ( Sx + b 1 m ) k } ( Sx + b 1 m ) k y k y k ! .
( x ; y , S , b ) = log p Y ( y ; x , S , b ) k = 1 m [ ( Sx ) k y k log ( Sx + b 1 m ) k ] ,
h ( t ) = i = 1 K a ( i ) δ ( t 2 d ( i ) / c ) , t [ 0 , T r ) ,
d s = min i = 1 , , K 1 | d ( i ) d ( i + 1 ) | .
minimize x ( x ; y , S , b ) + τ x 1 subject to x k 0 , k = 1 , 2 , , n ,
x ( x ; y , S , b ) = k = 1 m x [ ( Sx ) k y k log ( Sx + b 1 m ) k ] = k = 1 m [ ( S T ) k y k ( Sx + b 1 m ) k ( S T ) k ] = S T [ 1 m div ( y , Sx + b 1 m ) ] ,
NRMSE ( { d 1 , d 2 } , { d ^ 1 , d ^ 2 } ) = 1 c T p / 2 E [ 1 2 ( ( d 1 d ^ 1 ) 2 + ( d 2 d ^ 2 ) 2 ) ] ,

Metrics