Abstract

Exploiting temporal information of light propagation captured at ultra-fast frame rates has enabled applications such as reconstruction of complex hidden geometry and vision through scattering media. However, these applications require high-dimensional and high-resolution transport data, which introduces significant performance and storage constraints. Additionally, due to different sources of noise in both captured and synthesized data, the signal becomes significantly degraded over time, compromising the quality of the results. In this work, we tackle these issues by proposing a method that extracts meaningful sets of features to accurately represent time-resolved light transport data. Our method reduces the size of time-resolved transport data up to a factor of 32, while significantly mitigating variance in both temporal and spatial dimensions.

© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Novel time-resolved camera based on compressed sensing

A. Farina, A. Candeo, A. Dalla Mora, A. Bassi, R. Lussana, F. Villa, G. Valentini, S. Arridge, and C. D’Andrea
Opt. Express 27(22) 31889-31899 (2019)

Snapshot spatial–temporal compressive imaging

Mu Qiao, Xuan Liu, and Xin Yuan
Opt. Lett. 45(7) 1659-1662 (2020)

Real-time optical 3D reconstruction based on Monte Carlo integration and recurrent CNNs denoising with the 3D light field display

Yuanhang Li, Xinzhu Sang, Shujun Xing, Yanxin Guan, Shenwu Yang, Duo Chen, Le Yang, and Binbin Yan
Opt. Express 27(16) 22198-22208 (2019)

References

  • View by:
  • |
  • |
  • |

  1. A. Jarabo, B. Masia, J. Marco, and D. Gutierrez, Vis. Inform. 1, 65 (2017).
    [Crossref]
  2. F. Heide, M. Hullin, J. Gregson, and W. Heidrich, ACM Trans. Graph. 32, 45 (2013).
    [Crossref]
  3. A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
    [Crossref]
  4. A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, Nat. Commun. 3, 1 (2012).
    [Crossref]
  5. X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
    [Crossref]
  6. A. Jarabo, J. Marco, A. Muñoz, R. Buisan, W. Jarosz, and D. Gutierrez, ACM Trans. Graph. 33, 1 (2014).
    [Crossref]
  7. J. Marco, I. Guillén, W. Jarosz, D. Gutierrez, and A. Jarabo, Computer Graphics Forum 38, 19 (2019).
    [Crossref]
  8. J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
    [Crossref]
  9. Q. Guo, I. Frosio, O. Gallo, T. Zickler, and J. Kautz, in The European Conference on Computer Vision (ECCV) (2018).
  10. V. Arellano, D. Gutierrez, and A. Jarabo, Opt. Express 25, 11574 (2017).
    [Crossref]
  11. G. K. Wallace, IEEE Trans. Consum. Electron. 38, xviii (1992).
    [Crossref]
  12. D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
    [Crossref]
  13. F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, Opt. Express 22, 26338 (2014).
    [Crossref]
  14. Y. Liang, M. Chen, Z. Huang, D. Gutierrez, A. Muñoz, and J. Marco, in ACM SIGGRAPH 2019 Posters (ACM, 2019), p. 33.
  15. M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos, ACM Trans. Graph. 33, 1 (2014).
    [Crossref]
  16. R. Ng, R. Ramamoorthi, and P. Hanrahan, ACM Trans. Graph. 22, 376 (2003).
    [Crossref]
  17. K. He, X. Zhang, S. Ren, and J. Sun, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778.

2019 (2)

J. Marco, I. Guillén, W. Jarosz, D. Gutierrez, and A. Jarabo, Computer Graphics Forum 38, 19 (2019).
[Crossref]

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

2017 (3)

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

V. Arellano, D. Gutierrez, and A. Jarabo, Opt. Express 25, 11574 (2017).
[Crossref]

A. Jarabo, B. Masia, J. Marco, and D. Gutierrez, Vis. Inform. 1, 65 (2017).
[Crossref]

2014 (4)

A. Jarabo, J. Marco, A. Muñoz, R. Buisan, W. Jarosz, and D. Gutierrez, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
[Crossref]

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, Opt. Express 22, 26338 (2014).
[Crossref]

M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

2013 (2)

F. Heide, M. Hullin, J. Gregson, and W. Heidrich, ACM Trans. Graph. 32, 45 (2013).
[Crossref]

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

2012 (1)

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, Nat. Commun. 3, 1 (2012).
[Crossref]

2003 (1)

R. Ng, R. Ramamoorthi, and P. Hanrahan, ACM Trans. Graph. 22, 376 (2003).
[Crossref]

1992 (1)

G. K. Wallace, IEEE Trans. Consum. Electron. 38, xviii (1992).
[Crossref]

Agrawal, A.

D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
[Crossref]

Arellano, V.

Barsi, C.

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

Bawendi, M.

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

Bawendi, M. G.

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, Nat. Commun. 3, 1 (2012).
[Crossref]

Buisan, R.

A. Jarabo, J. Marco, A. Muñoz, R. Buisan, W. Jarosz, and D. Gutierrez, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

Chen, M.

Y. Liang, M. Chen, Z. Huang, D. Gutierrez, A. Muñoz, and J. Marco, in ACM SIGGRAPH 2019 Posters (ACM, 2019), p. 33.

Dai, Q.

D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
[Crossref]

Dong, Y.

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

Frosio, I.

Q. Guo, I. Frosio, O. Gallo, T. Zickler, and J. Kautz, in The European Conference on Computer Vision (ECCV) (2018).

Gallo, O.

Q. Guo, I. Frosio, O. Gallo, T. Zickler, and J. Kautz, in The European Conference on Computer Vision (ECCV) (2018).

Gregson, J.

F. Heide, M. Hullin, J. Gregson, and W. Heidrich, ACM Trans. Graph. 32, 45 (2013).
[Crossref]

Guillén, I.

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

J. Marco, I. Guillén, W. Jarosz, D. Gutierrez, and A. Jarabo, Computer Graphics Forum 38, 19 (2019).
[Crossref]

Guo, Q.

Q. Guo, I. Frosio, O. Gallo, T. Zickler, and J. Kautz, in The European Conference on Computer Vision (ECCV) (2018).

Gupta, O.

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, Nat. Commun. 3, 1 (2012).
[Crossref]

Gutierrez, D.

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

J. Marco, I. Guillén, W. Jarosz, D. Gutierrez, and A. Jarabo, Computer Graphics Forum 38, 19 (2019).
[Crossref]

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

A. Jarabo, B. Masia, J. Marco, and D. Gutierrez, Vis. Inform. 1, 65 (2017).
[Crossref]

V. Arellano, D. Gutierrez, and A. Jarabo, Opt. Express 25, 11574 (2017).
[Crossref]

A. Jarabo, J. Marco, A. Muñoz, R. Buisan, W. Jarosz, and D. Gutierrez, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

Y. Liang, M. Chen, Z. Huang, D. Gutierrez, A. Muñoz, and J. Marco, in ACM SIGGRAPH 2019 Posters (ACM, 2019), p. 33.

Hanrahan, P.

R. Ng, R. Ramamoorthi, and P. Hanrahan, ACM Trans. Graph. 22, 376 (2003).
[Crossref]

He, K.

K. He, X. Zhang, S. Ren, and J. Sun, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778.

Heide, F.

M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, Opt. Express 22, 26338 (2014).
[Crossref]

F. Heide, M. Hullin, J. Gregson, and W. Heidrich, ACM Trans. Graph. 32, 45 (2013).
[Crossref]

Heidrich, W.

M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, Opt. Express 22, 26338 (2014).
[Crossref]

F. Heide, M. Hullin, J. Gregson, and W. Heidrich, ACM Trans. Graph. 32, 45 (2013).
[Crossref]

Hernandez, Q.

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

Huang, Z.

Y. Liang, M. Chen, Z. Huang, D. Gutierrez, A. Muñoz, and J. Marco, in ACM SIGGRAPH 2019 Posters (ACM, 2019), p. 33.

Hullin, M.

F. Heide, M. Hullin, J. Gregson, and W. Heidrich, ACM Trans. Graph. 32, 45 (2013).
[Crossref]

Hullin, M. B.

M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, Opt. Express 22, 26338 (2014).
[Crossref]

Jarabo, A.

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

J. Marco, I. Guillén, W. Jarosz, D. Gutierrez, and A. Jarabo, Computer Graphics Forum 38, 19 (2019).
[Crossref]

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

A. Jarabo, B. Masia, J. Marco, and D. Gutierrez, Vis. Inform. 1, 65 (2017).
[Crossref]

V. Arellano, D. Gutierrez, and A. Jarabo, Opt. Express 25, 11574 (2017).
[Crossref]

A. Jarabo, J. Marco, A. Muñoz, R. Buisan, W. Jarosz, and D. Gutierrez, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

Jarosz, W.

J. Marco, I. Guillén, W. Jarosz, D. Gutierrez, and A. Jarabo, Computer Graphics Forum 38, 19 (2019).
[Crossref]

A. Jarabo, J. Marco, A. Muñoz, R. Buisan, W. Jarosz, and D. Gutierrez, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

Joshi, C.

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

Kautz, J.

Q. Guo, I. Frosio, O. Gallo, T. Zickler, and J. Kautz, in The European Conference on Computer Vision (ECCV) (2018).

Kim, M.

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

Kolb, A.

Kutulakos, K. N.

M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

La Manna, M.

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

Lawson, E.

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

Le, T. H.

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

Liang, Y.

Y. Liang, M. Chen, Z. Huang, D. Gutierrez, A. Muñoz, and J. Marco, in ACM SIGGRAPH 2019 Posters (ACM, 2019), p. 33.

Liu, X.

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

Marco, J.

J. Marco, I. Guillén, W. Jarosz, D. Gutierrez, and A. Jarabo, Computer Graphics Forum 38, 19 (2019).
[Crossref]

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

A. Jarabo, B. Masia, J. Marco, and D. Gutierrez, Vis. Inform. 1, 65 (2017).
[Crossref]

A. Jarabo, J. Marco, A. Muñoz, R. Buisan, W. Jarosz, and D. Gutierrez, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

Y. Liang, M. Chen, Z. Huang, D. Gutierrez, A. Muñoz, and J. Marco, in ACM SIGGRAPH 2019 Posters (ACM, 2019), p. 33.

Masia, B.

A. Jarabo, B. Masia, J. Marco, and D. Gutierrez, Vis. Inform. 1, 65 (2017).
[Crossref]

D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
[Crossref]

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

Muñoz, A.

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

A. Jarabo, J. Marco, A. Muñoz, R. Buisan, W. Jarosz, and D. Gutierrez, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

Y. Liang, M. Chen, Z. Huang, D. Gutierrez, A. Muñoz, and J. Marco, in ACM SIGGRAPH 2019 Posters (ACM, 2019), p. 33.

Nam, J. H.

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

Ng, R.

R. Ng, R. Ramamoorthi, and P. Hanrahan, ACM Trans. Graph. 22, 376 (2003).
[Crossref]

O’Toole, M.

M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
[Crossref]

Ramamoorthi, R.

R. Ng, R. Ramamoorthi, and P. Hanrahan, ACM Trans. Graph. 22, 376 (2003).
[Crossref]

Raskar, R.

D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
[Crossref]

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, Nat. Commun. 3, 1 (2012).
[Crossref]

Ren, S.

K. He, X. Zhang, S. Ren, and J. Sun, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778.

Reza, S. A.

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

Sun, J.

K. He, X. Zhang, S. Ren, and J. Sun, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778.

Tong, X.

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

Veeraraghavan, A.

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, Nat. Commun. 3, 1 (2012).
[Crossref]

Velten, A.

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
[Crossref]

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, Nat. Commun. 3, 1 (2012).
[Crossref]

Wallace, G. K.

G. K. Wallace, IEEE Trans. Consum. Electron. 38, xviii (1992).
[Crossref]

Willwacher, T.

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, Nat. Commun. 3, 1 (2012).
[Crossref]

Wu, D.

D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
[Crossref]

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

Xiao, L.

M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

F. Heide, L. Xiao, A. Kolb, M. B. Hullin, and W. Heidrich, Opt. Express 22, 26338 (2014).
[Crossref]

Zhang, X.

K. He, X. Zhang, S. Ren, and J. Sun, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778.

Zickler, T.

Q. Guo, I. Frosio, O. Gallo, T. Zickler, and J. Kautz, in The European Conference on Computer Vision (ECCV) (2018).

ACM Trans. Graph. (6)

F. Heide, M. Hullin, J. Gregson, and W. Heidrich, ACM Trans. Graph. 32, 45 (2013).
[Crossref]

A. Velten, D. Wu, A. Jarabo, B. Masia, C. Barsi, C. Joshi, E. Lawson, M. Bawendi, D. Gutierrez, and R. Raskar, ACM Trans. Graph. 32, 1 (2013).
[Crossref]

A. Jarabo, J. Marco, A. Muñoz, R. Buisan, W. Jarosz, and D. Gutierrez, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

J. Marco, Q. Hernandez, A. Muñoz, Y. Dong, A. Jarabo, M. Kim, X. Tong, and D. Gutierrez, ACM Trans. Graph. 36, 1 (2017).
[Crossref]

M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos, ACM Trans. Graph. 33, 1 (2014).
[Crossref]

R. Ng, R. Ramamoorthi, and P. Hanrahan, ACM Trans. Graph. 22, 376 (2003).
[Crossref]

Computer Graphics Forum (1)

J. Marco, I. Guillén, W. Jarosz, D. Gutierrez, and A. Jarabo, Computer Graphics Forum 38, 19 (2019).
[Crossref]

IEEE Trans. Consum. Electron. (1)

G. K. Wallace, IEEE Trans. Consum. Electron. 38, xviii (1992).
[Crossref]

Int. J. Comput. Vis. (1)

D. Wu, A. Velten, M. O’Toole, B. Masia, A. Agrawal, Q. Dai, and R. Raskar, Int. J. Comput. Vis. 107, 123 (2014).
[Crossref]

Nat. Commun. (1)

A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, Nat. Commun. 3, 1 (2012).
[Crossref]

Nature (1)

X. Liu, I. Guillén, M. La Manna, J. H. Nam, S. A. Reza, T. H. Le, A. Jarabo, D. Gutierrez, and A. Velten, Nature 572, 620 (2019).
[Crossref]

Opt. Express (2)

Vis. Inform. (1)

A. Jarabo, B. Masia, J. Marco, and D. Gutierrez, Vis. Inform. 1, 65 (2017).
[Crossref]

Other (3)

K. He, X. Zhang, S. Ren, and J. Sun, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778.

Q. Guo, I. Frosio, O. Gallo, T. Zickler, and J. Kautz, in The European Conference on Computer Vision (ECCV) (2018).

Y. Liang, M. Chen, Z. Huang, D. Gutierrez, A. Muñoz, and J. Marco, in ACM SIGGRAPH 2019 Posters (ACM, 2019), p. 33.

Supplementary Material (1)

NameDescription
» Visualization 1       Complete video sequences comparing reference and our decompressed results in Figures 4, 5, and 6.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1.
Fig. 1. Left: simulated steady-state render of the Altar scene. Middle: transient image of the scene. Right: time-resolved radiance at marked points of the scene.
Fig. 2.
Fig. 2. Our proposed architecture. The encoder extracts a total of $ T/32 $ features $ {f_L} $ from a $ 9 \times 9 \times T $ spatial neighborhood in logarithmic space $ {X^\prime } = g(\langle {L_{ij}}\rangle ) $ , centered at the time-resolved pixel $ {L_{ij}} $ to compress. The decoding step uses these features to recover the time-resolved pixel $ {\hat L_{ij}} = {g^{ - 1}}({Y^\prime }) $ with a set of deconvolutions and residual convolution blocks.
Fig. 3.
Fig. 3. Our encoder generates features that result in equally shaped radiance patterns at bounded time intervals of the decoder output (bottom left), which are then combined over the temporal domain in an overlapped manner (bottom right).
Fig. 4.
Fig. 4. Results of the Altar scene (see Visualization 1), with reference frames (left). Training with our exponential transform MSE loss [center, Eq. (4)] is able to recover strong direct peaks, while a MSE loss applied over the logarithmic-space of the output (right) fails to recover these features.
Fig. 5.
Fig. 5. Higher-order indirect illumination results in the Balcony (left) and Building (right) scenes from the validation set. Images show selected reference frames (top) and our denoised frames (bottom) after encoding and decoding each time-resolved pixel. Plots show our time-resolved profiles at marked pixels (green), reference samples of that pixel (blue), compressed features (red), and all the spatiotemporal input samples analyzed by our encoder, color-coded by the distance to the center of the neighborhood $ \langle {L_{ij}}\rangle $ .
Fig. 6.
Fig. 6. Room scene results, showing high decompression accuracy, recovering both high- and low-frequency features in the temporal domain. Top: reference frames. Center: our resulting frames after decompression. Bottom: time-resolved transport at the marked location (top-left).
Fig. 7.
Fig. 7. Results for real data (blue) captured on a non-line-of-sight setup (left) [5]. The plots show our results (green) at different points of the captured grid.

Tables (1)

Tables Icon

Table 1. Reduction Ratios for the Validation Scenes Illustrated in This Letter for Standard Libraries Supporting HDR Compression

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

E ( X ) = f L , w h e r e X = g ( X ) .
D ( f L ) = Y , w h e r e g 1 ( Y ) = L ^ i j ( t ) ,
g ( X ) = { log 10 ( X ) log 10 ( ε ) X ε 0 X < ε .
L = 1 T t = 0 T 1 ( b g ( L i j ( t ) ) b D ( f L ) ( t ) ) 2 ,