Abstract

A remote-sensing system that can determine the position of hidden objects has applications in many critical real-life scenarios, such as search and rescue missions and safe autonomous driving. Previous work has shown the ability to range and image objects hidden from the direct line of sight, employing advanced optical imaging technologies aimed at small objects at short range. In this work we demonstrate a long-range tracking system based on single laser illumination and single-pixel single-photon detection. This enables us to track one or more people hidden from view at a stand-off distance of over 50 m. These results pave the way towards next generation LiDAR systems that will reconstruct not only the direct-view scene but also the main elements hidden behind walls or corners.

Published by The Optical Society under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Full Article  |  PDF Article
OSA Recommended Articles
Non-line-of-sight imaging using a time-gated single photon avalanche diode

Mauro Buttafava, Jessica Zeman, Alberto Tosi, Kevin Eliceiri, and Andreas Velten
Opt. Express 23(16) 20997-21011 (2015)

Fast back-projection for non-line of sight reconstruction

Victor Arellano, Diego Gutierrez, and Adrian Jarabo
Opt. Express 25(10) 11574-11583 (2017)

Reconstruction of multiple non-line-of-sight objects using back projection based on ellipsoid mode decomposition

Chenfei Jin, Jiaheng Xie, Siqi Zhang, ZiJing Zhang, and Yuan Zhao
Opt. Express 26(16) 20089-20101 (2018)

References

  • View by:
  • |
  • |
  • |

  1. A. Kirmani, T. Hutchison, J. Davis, and R. Raskar, “Looking around the corner using transient imaging,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 159–166.
  2. R. Pandharkar, A. Velten, A. Bardagjy, E. Lawson, M. Bawendi, and R. Raskar, “Estimating motion and size of moving non-line-of-sight objects in cluttered environments,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 265–272.
  3. A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012).
    [Crossref]
  4. O. Gupta, T. Willwacher, A. Velten, A. Veeraraghavan, and R. Raskar, “Reconstruction of hidden 3D shapes using diffuse reflections,” Opt. Express 20, 19096–19108 (2012).
    [Crossref] [PubMed]
  5. F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 45–50 (2013).
    [Crossref]
  6. M. Buttafava, J. Zeman, A. Tosi, K. Eliceiri, and A. Velten, “Non-line-of-sight imaging using a time-gated single photon avalanche diode,” Opt. Express 23, 20997–21115 (2015).
    [Crossref] [PubMed]
  7. M. Laurenzis, J. Klein, E. Bacher, and N. Metzger, “Multiple-return single-photon counting of light in flight and sensing of non-line-of-sight objects at shortwave infrared wavelengths,” Opt. Lett. 40, 4815–4818 (2015).
    [Crossref] [PubMed]
  8. G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nature Photon. 10, 23–26 (2015).
    [Crossref]
  9. S. Shrestha, F. Heide, W. Heidrich, and G. Wetzstein, “Computational imaging with multi-camera time-of-flight systems,” ACM Trans. Graph. 35, 1–11 (2016).
    [Crossref]
  10. A. Kadambi, H. Zhao, B. Shi, and R. Raskar, “Occluded imaging with time-of-flight sensors,” ACM Trans. Graph. 35, 1–12 (2016).
    [Crossref]
  11. J. Klein, C. Peters, J. Martín, and M. Laurenzis, “Tracking objects outside the line of sight using 2D intensity images,” Scientific Reports 6, 32491 (2016).
    [Crossref] [PubMed]
  12. U. Wandinger, “Introduction to Lidar,” in Lidar: Range-Resolved Optical Remote Sensing of the Atmosphere, C. Weitkamp, ed. (Springer, 2005), pp. 1–18.
    [Crossref]
  13. C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
    [Crossref]
  14. F. Zappa, S. Tisa, A. Tosi, and S. Cova, “Principles and features of single-photon avalanche diode arrays,” Sens. Actuators A Phys. 140, 103–112 (2007).
    [Crossref]
  15. J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.
  16. E. Charbon, M. Fishburn, R. Walker, R. K. Henderson, and C. Niclass, “SPAD-Based Sensors,” in “TOF Range-Imaging Cameras,” F. Remondino and D. Stoppa, eds. (Springer, 2013), pp. 11–38.
    [Crossref]
  17. J. A. Richardson, L. A. Grant, and R. K. Henderson, “Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology,” IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).
    [Crossref]
  18. G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
    [Crossref]
  19. W. Becker, Advanced Time-Correlated Single Photon Counting Techniques (Springer, 2005).
    [Crossref]
  20. R. Cutler and L. Davis, “View-based detection and analysis of periodic motion,” in Proceedings of International Conference on Pattern Recognition (IEEE Comput. Soc, 1998), pp. 495–500.
  21. R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1337–1342 (2003).
    [Crossref]
  22. T. S. Ralston, G. L. Charvat, and J. E. Peabody, “Real-time through-wall imaging using an ultrawideband multiple-input multiple-output (MIMO) phased array radar system,” in Proceedings of IEEE International Symposium on Phased Array Systems and Technology (IEEE, 2010), pp. 551–558.

2016 (3)

S. Shrestha, F. Heide, W. Heidrich, and G. Wetzstein, “Computational imaging with multi-camera time-of-flight systems,” ACM Trans. Graph. 35, 1–11 (2016).
[Crossref]

A. Kadambi, H. Zhao, B. Shi, and R. Raskar, “Occluded imaging with time-of-flight sensors,” ACM Trans. Graph. 35, 1–12 (2016).
[Crossref]

J. Klein, C. Peters, J. Martín, and M. Laurenzis, “Tracking objects outside the line of sight using 2D intensity images,” Scientific Reports 6, 32491 (2016).
[Crossref] [PubMed]

2015 (4)

G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nature Photon. 10, 23–26 (2015).
[Crossref]

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

M. Buttafava, J. Zeman, A. Tosi, K. Eliceiri, and A. Velten, “Non-line-of-sight imaging using a time-gated single photon avalanche diode,” Opt. Express 23, 20997–21115 (2015).
[Crossref] [PubMed]

M. Laurenzis, J. Klein, E. Bacher, and N. Metzger, “Multiple-return single-photon counting of light in flight and sensing of non-line-of-sight objects at shortwave infrared wavelengths,” Opt. Lett. 40, 4815–4818 (2015).
[Crossref] [PubMed]

2013 (1)

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 45–50 (2013).
[Crossref]

2012 (2)

O. Gupta, T. Willwacher, A. Velten, A. Veeraraghavan, and R. Raskar, “Reconstruction of hidden 3D shapes using diffuse reflections,” Opt. Express 20, 19096–19108 (2012).
[Crossref] [PubMed]

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012).
[Crossref]

2009 (1)

J. A. Richardson, L. A. Grant, and R. K. Henderson, “Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology,” IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).
[Crossref]

2007 (1)

F. Zappa, S. Tisa, A. Tosi, and S. Cova, “Principles and features of single-photon avalanche diode arrays,” Sens. Actuators A Phys. 140, 103–112 (2007).
[Crossref]

2005 (1)

C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

2003 (1)

R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1337–1342 (2003).
[Crossref]

Bacher, E.

Bardagjy, A.

R. Pandharkar, A. Velten, A. Bardagjy, E. Lawson, M. Bawendi, and R. Raskar, “Estimating motion and size of moving non-line-of-sight objects in cluttered environments,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 265–272.

Bawendi, M.

R. Pandharkar, A. Velten, A. Bardagjy, E. Lawson, M. Bawendi, and R. Raskar, “Estimating motion and size of moving non-line-of-sight objects in cluttered environments,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 265–272.

Bawendi, M. G.

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012).
[Crossref]

Becker, W.

W. Becker, Advanced Time-Correlated Single Photon Counting Techniques (Springer, 2005).
[Crossref]

Besse, P.-A.

C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

Borghetti, F.

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.

Buller, G. S.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

Buttafava, M.

Charbon, E.

C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.

E. Charbon, M. Fishburn, R. Walker, R. K. Henderson, and C. Niclass, “SPAD-Based Sensors,” in “TOF Range-Imaging Cameras,” F. Remondino and D. Stoppa, eds. (Springer, 2013), pp. 11–38.
[Crossref]

Charvat, G. L.

T. S. Ralston, G. L. Charvat, and J. E. Peabody, “Real-time through-wall imaging using an ultrawideband multiple-input multiple-output (MIMO) phased array radar system,” in Proceedings of IEEE International Symposium on Phased Array Systems and Technology (IEEE, 2010), pp. 551–558.

Cova, S.

F. Zappa, S. Tisa, A. Tosi, and S. Cova, “Principles and features of single-photon avalanche diode arrays,” Sens. Actuators A Phys. 140, 103–112 (2007).
[Crossref]

Cucchiara, R.

R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1337–1342 (2003).
[Crossref]

Cutler, R.

R. Cutler and L. Davis, “View-based detection and analysis of periodic motion,” in Proceedings of International Conference on Pattern Recognition (IEEE Comput. Soc, 1998), pp. 495–500.

Davis, J.

A. Kirmani, T. Hutchison, J. Davis, and R. Raskar, “Looking around the corner using transient imaging,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 159–166.

Davis, L.

R. Cutler and L. Davis, “View-based detection and analysis of periodic motion,” in Proceedings of International Conference on Pattern Recognition (IEEE Comput. Soc, 1998), pp. 495–500.

Eliceiri, K.

Faccio, D.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nature Photon. 10, 23–26 (2015).
[Crossref]

Fishburn, M.

E. Charbon, M. Fishburn, R. Walker, R. K. Henderson, and C. Niclass, “SPAD-Based Sensors,” in “TOF Range-Imaging Cameras,” F. Remondino and D. Stoppa, eds. (Springer, 2013), pp. 11–38.
[Crossref]

Gariepy, G.

G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nature Photon. 10, 23–26 (2015).
[Crossref]

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

Gersbach, M.

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.

Grana, C.

R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1337–1342 (2003).
[Crossref]

Grant, L.

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.

Grant, L. A.

J. A. Richardson, L. A. Grant, and R. K. Henderson, “Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology,” IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).
[Crossref]

Gregson, J.

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 45–50 (2013).
[Crossref]

Gupta, O.

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012).
[Crossref]

O. Gupta, T. Willwacher, A. Velten, A. Veeraraghavan, and R. Raskar, “Reconstruction of hidden 3D shapes using diffuse reflections,” Opt. Express 20, 19096–19108 (2012).
[Crossref] [PubMed]

Heide, F.

S. Shrestha, F. Heide, W. Heidrich, and G. Wetzstein, “Computational imaging with multi-camera time-of-flight systems,” ACM Trans. Graph. 35, 1–11 (2016).
[Crossref]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 45–50 (2013).
[Crossref]

Heidrich, W.

S. Shrestha, F. Heide, W. Heidrich, and G. Wetzstein, “Computational imaging with multi-camera time-of-flight systems,” ACM Trans. Graph. 35, 1–11 (2016).
[Crossref]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 45–50 (2013).
[Crossref]

Henderson, R.

G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nature Photon. 10, 23–26 (2015).
[Crossref]

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

Henderson, R. K.

J. A. Richardson, L. A. Grant, and R. K. Henderson, “Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology,” IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).
[Crossref]

E. Charbon, M. Fishburn, R. Walker, R. K. Henderson, and C. Niclass, “SPAD-Based Sensors,” in “TOF Range-Imaging Cameras,” F. Remondino and D. Stoppa, eds. (Springer, 2013), pp. 11–38.
[Crossref]

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.

Heshmat, B.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

Hullin, M. B.

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 45–50 (2013).
[Crossref]

Hutchison, T.

A. Kirmani, T. Hutchison, J. Davis, and R. Raskar, “Looking around the corner using transient imaging,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 159–166.

Kadambi, A.

A. Kadambi, H. Zhao, B. Shi, and R. Raskar, “Occluded imaging with time-of-flight sensors,” ACM Trans. Graph. 35, 1–12 (2016).
[Crossref]

Kirmani, A.

A. Kirmani, T. Hutchison, J. Davis, and R. Raskar, “Looking around the corner using transient imaging,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 159–166.

Klein, J.

Krstajic, N.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

Laurenzis, M.

Lawson, E.

R. Pandharkar, A. Velten, A. Bardagjy, E. Lawson, M. Bawendi, and R. Raskar, “Estimating motion and size of moving non-line-of-sight objects in cluttered environments,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 265–272.

Leach, J.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nature Photon. 10, 23–26 (2015).
[Crossref]

Li, C.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

Martín, J.

J. Klein, C. Peters, J. Martín, and M. Laurenzis, “Tracking objects outside the line of sight using 2D intensity images,” Scientific Reports 6, 32491 (2016).
[Crossref] [PubMed]

Metzger, N.

Niclass, C.

C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

E. Charbon, M. Fishburn, R. Walker, R. K. Henderson, and C. Niclass, “SPAD-Based Sensors,” in “TOF Range-Imaging Cameras,” F. Remondino and D. Stoppa, eds. (Springer, 2013), pp. 11–38.
[Crossref]

Pandharkar, R.

R. Pandharkar, A. Velten, A. Bardagjy, E. Lawson, M. Bawendi, and R. Raskar, “Estimating motion and size of moving non-line-of-sight objects in cluttered environments,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 265–272.

Peabody, J. E.

T. S. Ralston, G. L. Charvat, and J. E. Peabody, “Real-time through-wall imaging using an ultrawideband multiple-input multiple-output (MIMO) phased array radar system,” in Proceedings of IEEE International Symposium on Phased Array Systems and Technology (IEEE, 2010), pp. 551–558.

Peters, C.

J. Klein, C. Peters, J. Martín, and M. Laurenzis, “Tracking objects outside the line of sight using 2D intensity images,” Scientific Reports 6, 32491 (2016).
[Crossref] [PubMed]

Piccardi, M.

R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1337–1342 (2003).
[Crossref]

Prati, A.

R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1337–1342 (2003).
[Crossref]

Ralston, T. S.

T. S. Ralston, G. L. Charvat, and J. E. Peabody, “Real-time through-wall imaging using an ultrawideband multiple-input multiple-output (MIMO) phased array radar system,” in Proceedings of IEEE International Symposium on Phased Array Systems and Technology (IEEE, 2010), pp. 551–558.

Raskar, R.

A. Kadambi, H. Zhao, B. Shi, and R. Raskar, “Occluded imaging with time-of-flight sensors,” ACM Trans. Graph. 35, 1–12 (2016).
[Crossref]

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

O. Gupta, T. Willwacher, A. Velten, A. Veeraraghavan, and R. Raskar, “Reconstruction of hidden 3D shapes using diffuse reflections,” Opt. Express 20, 19096–19108 (2012).
[Crossref] [PubMed]

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012).
[Crossref]

A. Kirmani, T. Hutchison, J. Davis, and R. Raskar, “Looking around the corner using transient imaging,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 159–166.

R. Pandharkar, A. Velten, A. Bardagjy, E. Lawson, M. Bawendi, and R. Raskar, “Estimating motion and size of moving non-line-of-sight objects in cluttered environments,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 265–272.

Richardson, J.

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.

Richardson, J. A.

J. A. Richardson, L. A. Grant, and R. K. Henderson, “Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology,” IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).
[Crossref]

Rochas, A.

C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

Shi, B.

A. Kadambi, H. Zhao, B. Shi, and R. Raskar, “Occluded imaging with time-of-flight sensors,” ACM Trans. Graph. 35, 1–12 (2016).
[Crossref]

Shrestha, S.

S. Shrestha, F. Heide, W. Heidrich, and G. Wetzstein, “Computational imaging with multi-camera time-of-flight systems,” ACM Trans. Graph. 35, 1–11 (2016).
[Crossref]

Stoppa, D.

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.

Thomson, R. R.

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

Tisa, S.

F. Zappa, S. Tisa, A. Tosi, and S. Cova, “Principles and features of single-photon avalanche diode arrays,” Sens. Actuators A Phys. 140, 103–112 (2007).
[Crossref]

Tonolini, F.

G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nature Photon. 10, 23–26 (2015).
[Crossref]

Tosi, A.

M. Buttafava, J. Zeman, A. Tosi, K. Eliceiri, and A. Velten, “Non-line-of-sight imaging using a time-gated single photon avalanche diode,” Opt. Express 23, 20997–21115 (2015).
[Crossref] [PubMed]

F. Zappa, S. Tisa, A. Tosi, and S. Cova, “Principles and features of single-photon avalanche diode arrays,” Sens. Actuators A Phys. 140, 103–112 (2007).
[Crossref]

Veeraraghavan, A.

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012).
[Crossref]

O. Gupta, T. Willwacher, A. Velten, A. Veeraraghavan, and R. Raskar, “Reconstruction of hidden 3D shapes using diffuse reflections,” Opt. Express 20, 19096–19108 (2012).
[Crossref] [PubMed]

Velten, A.

M. Buttafava, J. Zeman, A. Tosi, K. Eliceiri, and A. Velten, “Non-line-of-sight imaging using a time-gated single photon avalanche diode,” Opt. Express 23, 20997–21115 (2015).
[Crossref] [PubMed]

O. Gupta, T. Willwacher, A. Velten, A. Veeraraghavan, and R. Raskar, “Reconstruction of hidden 3D shapes using diffuse reflections,” Opt. Express 20, 19096–19108 (2012).
[Crossref] [PubMed]

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012).
[Crossref]

R. Pandharkar, A. Velten, A. Bardagjy, E. Lawson, M. Bawendi, and R. Raskar, “Estimating motion and size of moving non-line-of-sight objects in cluttered environments,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 265–272.

Walker, R.

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.

E. Charbon, M. Fishburn, R. Walker, R. K. Henderson, and C. Niclass, “SPAD-Based Sensors,” in “TOF Range-Imaging Cameras,” F. Remondino and D. Stoppa, eds. (Springer, 2013), pp. 11–38.
[Crossref]

Wandinger, U.

U. Wandinger, “Introduction to Lidar,” in Lidar: Range-Resolved Optical Remote Sensing of the Atmosphere, C. Weitkamp, ed. (Springer, 2005), pp. 1–18.
[Crossref]

Wetzstein, G.

S. Shrestha, F. Heide, W. Heidrich, and G. Wetzstein, “Computational imaging with multi-camera time-of-flight systems,” ACM Trans. Graph. 35, 1–11 (2016).
[Crossref]

Willwacher, T.

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012).
[Crossref]

O. Gupta, T. Willwacher, A. Velten, A. Veeraraghavan, and R. Raskar, “Reconstruction of hidden 3D shapes using diffuse reflections,” Opt. Express 20, 19096–19108 (2012).
[Crossref] [PubMed]

Zappa, F.

F. Zappa, S. Tisa, A. Tosi, and S. Cova, “Principles and features of single-photon avalanche diode arrays,” Sens. Actuators A Phys. 140, 103–112 (2007).
[Crossref]

Zeman, J.

Zhao, H.

A. Kadambi, H. Zhao, B. Shi, and R. Raskar, “Occluded imaging with time-of-flight sensors,” ACM Trans. Graph. 35, 1–12 (2016).
[Crossref]

ACM Trans. Graph. (3)

S. Shrestha, F. Heide, W. Heidrich, and G. Wetzstein, “Computational imaging with multi-camera time-of-flight systems,” ACM Trans. Graph. 35, 1–11 (2016).
[Crossref]

A. Kadambi, H. Zhao, B. Shi, and R. Raskar, “Occluded imaging with time-of-flight sensors,” ACM Trans. Graph. 35, 1–12 (2016).
[Crossref]

F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 45–50 (2013).
[Crossref]

IEEE J. Solid-State Circuits (1)

C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005).
[Crossref]

IEEE Photon. Technol. Lett. (1)

J. A. Richardson, L. A. Grant, and R. K. Henderson, “Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology,” IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1337–1342 (2003).
[Crossref]

Nature Comms (2)

G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015).
[Crossref]

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012).
[Crossref]

Nature Photon. (1)

G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nature Photon. 10, 23–26 (2015).
[Crossref]

Opt. Express (2)

Opt. Lett. (1)

Scientific Reports (1)

J. Klein, C. Peters, J. Martín, and M. Laurenzis, “Tracking objects outside the line of sight using 2D intensity images,” Scientific Reports 6, 32491 (2016).
[Crossref] [PubMed]

Sens. Actuators A Phys. (1)

F. Zappa, S. Tisa, A. Tosi, and S. Cova, “Principles and features of single-photon avalanche diode arrays,” Sens. Actuators A Phys. 140, 103–112 (2007).
[Crossref]

Other (8)

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80.

E. Charbon, M. Fishburn, R. Walker, R. K. Henderson, and C. Niclass, “SPAD-Based Sensors,” in “TOF Range-Imaging Cameras,” F. Remondino and D. Stoppa, eds. (Springer, 2013), pp. 11–38.
[Crossref]

W. Becker, Advanced Time-Correlated Single Photon Counting Techniques (Springer, 2005).
[Crossref]

R. Cutler and L. Davis, “View-based detection and analysis of periodic motion,” in Proceedings of International Conference on Pattern Recognition (IEEE Comput. Soc, 1998), pp. 495–500.

U. Wandinger, “Introduction to Lidar,” in Lidar: Range-Resolved Optical Remote Sensing of the Atmosphere, C. Weitkamp, ed. (Springer, 2005), pp. 1–18.
[Crossref]

A. Kirmani, T. Hutchison, J. Davis, and R. Raskar, “Looking around the corner using transient imaging,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 159–166.

R. Pandharkar, A. Velten, A. Bardagjy, E. Lawson, M. Bawendi, and R. Raskar, “Estimating motion and size of moving non-line-of-sight objects in cluttered environments,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 265–272.

T. S. Ralston, G. L. Charvat, and J. E. Peabody, “Real-time through-wall imaging using an ultrawideband multiple-input multiple-output (MIMO) phased array radar system,” in Proceedings of IEEE International Symposium on Phased Array Systems and Technology (IEEE, 2010), pp. 551–558.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1 Looking down the corridor. We perform our experiment by looking down the corridor at a screen. (a) shows a schematic overview of the geometry of the environment. The blind corners obstruct direct vision of the hidden person(s). In a single measurement, we illuminate a single spot on the wall with a high-repetition pulsed laser and detect the light scattered back to the points on the wall which we image to our SPAD detector. “L” indicates the position of the laser on the screen and the numbered squares indicate positions imaged (one at a time) onto the single-pixel SPAD. (b) shows a photograph of the environment. Lighting conditions are as in the experiment. (c) shows a photograph of the equipment.
Fig. 2
Fig. 2 Target-position retrieval. (a) Laser pulses incident on the wall scatter and propagate approximately as an isotropic spherical wavefront. Light reaching an object scatters back in a similar fashion. The inset histogram shows an example of the signal recorded by the single-pixel detector. (b) The time extracted from the peaks (of the Gaussian fits to the data, shown in the inset) provide the total distance travelled by the light, from the wall to the person and back to the wall, but not the actual path taken. The bold and dashed lines in the schematic show two equivalent paths, corresponding to the time extracted from the histogram for one pixel position - there are infinite such paths that combined, form an ellipsoid of equally probable locations for the hidden object. By repeating the measurement four times (looking at different positions on the screen as shown in Fig. 1(a)) we obtain four ellipses that overlap in correspondence to the target position in two dimensions.
Fig. 3
Fig. 3 Experimental results for non-line-of-sight tracking of a single person. We perform tracking of a hidden person at distinct positions around a blind corner. Each coloured curve in the graph is a joint probability distribution of the person’s retrieved position, while the corresponding rectangle is their actual position during the measurement.
Fig. 4
Fig. 4 Simulation setup for a two-detector system. To investigate the baseline distance between the detector positions on the screen, we simulate an experiment with a single laser and two detectors. (a) and (b) show a schematic overview of the simulated environment in top-down and perspective view respectively. Laser position “L” and pixel position “D1” are fixed while pixel position “D2” is scanned in the x-direction. “Oi” indicates the location of the hidden object in each set of simulated measurements.
Fig. 5
Fig. 5 Simulation results for a two-detector system. We investigate how separating out the pixels in one direction affects the accuracy and precision of the target position retrieval. (a) and (b) show the respective change in accuracy of the retrieved x- and y-coordinates (“errorx” and “errory”) as the separation between pixels increases, while (c) and (d) show the change in precision in the x- and y-directions (“σx” and “σy”) respectively.
Fig. 6
Fig. 6 Experimental results for tracking of multiple hidden persons. We perform tracking of two people located around the same blind corner. (a) and (b) show the raw data corresponding to just one of the four pixel positions used to identify and locate the two people, as shown in (c) and (d).
Fig. 7
Fig. 7 Experimental results for detecting multiple hidden persons in less favourable conditions. We perform detection of two hidden persons, each located around either corner of the T-junction. The coloured curves and corresponding rectangles show each person’s retrieved position and actual position relative to the righthand-side corner.

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

P i ( r o ) exp [ ( | r o r l | + | r o r i | c t i ) 2 2 σ i 2 ]

Metrics