Abstract

Quanta Image Sensor (QIS) is a single-photon detector designed for extremely low light imaging conditions. Majority of the existing QIS prototypes are monochrome based on single-photon avalanche diodes (SPAD). Passive color imaging has not been demonstrated with single-photon detectors due to the intrinsic difficulty of shrinking the pixel size and increasing the spatial resolution while maintaining acceptable intra-pixel cross-talk. In this paper, we present image reconstruction of the first color QIS with a resolution of 1024 × 1024 pixels, supporting both single-bit and multi-bit photon counting capability. Our color image reconstruction is enabled by a customized joint demosaicing-denoising algorithm, leveraging truncated Poisson statistics andvariance stabilizing transforms. Experimental results of the new sensor and algorithm demonstrate superior color imaging performance for very low-light conditions with a mean exposure of as low as a few photons per pixel in both real and simulated images.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Application of the Quanta image sensor concept to linear polarization imaging—a theoretical study

Leo Anzagira and Eric R. Fossum
J. Opt. Soc. Am. A 33(6) 1147-1154 (2016)

High-resolution depth profiling using a range-gated CMOS SPAD quanta image sensor

Ximing Ren, Peter W. R. Connolly, Abderrahim Halimi, Yoann Altmann, Stephen McLaughlin, Istvan Gyongy, Robert K. Henderson, and Gerald S. Buller
Opt. Express 26(5) 5541-5557 (2018)

Photon-number-resolving megapixel image sensor at room temperature without avalanche gain

Jiaju Ma, Saleh Masoodian, Dakota A. Starkey, and Eric R. Fossum
Optica 4(12) 1474-1481 (2017)

References

  • View by:
  • |
  • |
  • |

  1. E. R. Fossum, “Some Thoughts on Future Digital Still Cameras,” in Image Sensors and Signal Processing for Digital Still Cameras, (CRC, 2006), 305–314.
  2. E. R. Fossum, “Gigapixel digital film sensor (DFS) proposal,” in Nanospace Manipulation of Photons and Electrons for Nanovision Systems, The 7th Takayanagi Kenjiro Memorial Symposium and the 2nd International Symposium on Nanovision Science, Hamamatsu, Japan, (2005).
  3. E. R. Fossum, “Active pixel sensors: Are CCDs dinosaurs?” in Charge-Coupled Devices and Solid State Optical Sensors III, vol. 1900 (International Society for Optics and Photonics, 1993), 2–15.
  4. J. Ma and E. R. Fossum, “A pump-gate jot device with high conversion gain for a quanta image sensor,” IEEE J. Electron Devices Soc. 3, 73–77 (2015).
  5. J. Ma, S. Masoodian, D. A. Starkey, and E. R. Fossum, “Photon-number-resolving megapixel image sensor at room temperature without avalanche gain,” Optica 4, 1474–1481 (2017).
    [Crossref]
  6. J. Hynecek, “Impactron-a new solid state image intensifier,” IEEE Trans. Electron Devices 48, 2238–2241 (2001).
    [Crossref]
  7. N. A. W. Dutton, L. Parmesan, A. J. Holmes, L. A. Grant, and R. K. Henderson, “320×240 oversampled digital single photon counting image sensor,” in Proceedings of IEEE 2014 Symposium on VLSI Circuits Digest of Technical Papers, (IEEE, 2014).
  8. C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).
  9. S. H. Chan and Y. M. Lu, “Efficient image reconstruction for gigapixel quantum image sensors,” in Proceedings of IEEE Global Conference on Signal and Information Processing, (IEEE, 2014), 312–316.
  10. S. H. Chan, O. A. Elgendy, and X. Wang, “Images from bits: Non-iterative image reconstruction for quanta image sensors,” Sensors 16, 1961 (2016).
    [Crossref] [PubMed]
  11. O. A. Elgendy and S. H. Chan, “Optimal threshold design for quanta image sensor,” IEEE Trans. Comput. Imag. 4, 99–111 (2018).
  12. F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “Bits from photons: Oversampled image acquisition using binary Poisson statistics,” IEEE Trans. Image Process. 21, 1421–1436 (2012).
  13. X. Ren, Y. Altmann, R. Tobin, A. Mccarthy, S. Mclaughlin, and G. S. Buller, “Wavelength-time coding for multispectral 3d imaging using single-photon lidar,” Opt.Express 26, 30146–30161 (2018).
  14. N. A. Dutton, I. Gyongy, L. Parmesan, and R. K. Henderson, “Single photon counting performance and noise analysis of CMOS SPAD-based image sensors,” Sensors 161122 (2016).
    [Crossref] [PubMed]
  15. I. Gyongy, N. A. Dutton, and R. K. Henderson, “Single-photon tracking for high-speed vision,” Sensors 18323 (2018).
    [Crossref]
  16. S. H. Chan, X. Wang, and O. A. Elgendy, “Plug-and-play ADMM for image restoration: Fixed-point convergence and applications,” IEEE Trans. Comput. Imag. 384–98 (2017).
  17. B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).
  18. J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
    [Crossref] [PubMed]
  19. E. Charbon, “Towards large scale CMOS single-photon detector arrays for lab-on-chip applications,” J. Phys. D: Appl. Phys. 41, 094010 (2008).
    [Crossref]
  20. E. Charbon, “Will avalanche photodiode arrays ever reach 1 megapixel,” in Proceedings of International Image Sensor Workshop, (IISS, 2007), 246–249.
  21. J. Ma and E. R. Fossum, “Quanta image sensor jot with sub 0.3 e-rms read noise and photon counting capability,” IEEE Electron Device Lett. 36, 926–928 (2015).
    [Crossref]
  22. S. Masoodian, J. Ma, D. Starkey, Y. Yamashita, and E. R. Fossum, “A 1Mjot 1040fps 0.22 e-rms stacked BSI quanta image sensor with Cluster-Parallel Readout,” in Proceedings of the International Image Sensor Workshop, vol. 30 (IISS, 2017), 230–233.
  23. N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
    [Crossref]
  24. “Zylus 4.2 PLUS,” https://andor.oxinst.com/products/scmos-camera-series/zyla-4-2-scmos . Accessed: 2019-04-21.
  25. F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “An optimal algorithm for reconstructing images from binary measurements,” in Computational Imaging VIII, vol. 7533 (International Society for Optics and Photonics, 2010), 75330K.
    [Crossref]
  26. T. Remez, O. Litany, and A. Bronstein, “A picture is worth a billion bits: Real-time image reconstruction from dense binary threshold pixels,” in Proceedings of IEEE International Conference on Computational Photography, (IEEE, 2016), 1–9.
  27. J. H. Choi, O. A. Elgendy, and S. H. Chan, “Image reconstruction for quanta image sensors using deep neural networks,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (IEEE, 2018), 6543–6547.
  28. P. Chandramouli, S. Burri, C. Bruschini, E. Charbon, and A. Kolb, “A little bit too much? High speed imaging from sparse photon counts,” arXiv preprint arXiv:1811.02396 (2018).
  29. R. A. Rojas, W. Luo, V. Murray, and Y. M. Lu, “Learning optimal parameters for binary sensing image reconstruction algorithms,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2017), 2791–2795.
  30. L. Azzari and A. Foi, “Variance stabilization in Poisson image deblurring,” in IEEE International Symposium on Biomedical Imaging, (IEEE, 2017), 728–731.
  31. L. Azzari and A. Foi, “Variance stabilization for noisy+ estimate combination in iterative Poisson denoising,” IEEE Signal Process. Lett. 23, 1086–1090 (2016).
    [Crossref]
  32. A. Foi, “Noise estimation and removal in MR imaging: The variance-stabilization approach,” in Proceedings of International symposium on biomedical imaging: from nano to macro, (IEEE, 2011), 1809–1814.
  33. F. J. Anscombe, “The transformation of Poisson, binomial and negative-binomial data,” Biometrika 35, 246–254 (1948).
    [Crossref]
  34. M. Makitalo and A. Foi, “Optimal inversion of the Anscombe transformation in low-count Poisson image denoising,” IEEE Trans. Image Process.,  20, 99–109 (2011).
    [Crossref]
  35. S. H. Chan, “Performance analysis of plug-and-play admm: A graph signal processing perspective,” IEEE Trans. Comput. Imag. (2019).
    [Crossref]
  36. G. Jeon and E. Dubois, “Demosaicking of noisy Bayer-sampled color images with least-squares Luma-Chroma demultiplexing and noise level estimation,” IEEE Trans. Image Process. 22, 146–156 (2013).
  37. J. T. Korneliussen and K. Hirakawa, “Camera processing with chromatic aberration,” IEEE Trans. on Image Process. 23, 4539– 4552 (2014).
    [Crossref]
  38. K. Hirakawa, X. Meng, and P. J. Wolfe, “A framework for wavelet-based analysis and processing of color filter array images with applications to denoising and demosaicing,” in IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1 (IEEE, 2007), 597–600.

2018 (4)

O. A. Elgendy and S. H. Chan, “Optimal threshold design for quanta image sensor,” IEEE Trans. Comput. Imag. 4, 99–111 (2018).

X. Ren, Y. Altmann, R. Tobin, A. Mccarthy, S. Mclaughlin, and G. S. Buller, “Wavelength-time coding for multispectral 3d imaging using single-photon lidar,” Opt.Express 26, 30146–30161 (2018).

I. Gyongy, N. A. Dutton, and R. K. Henderson, “Single-photon tracking for high-speed vision,” Sensors 18323 (2018).
[Crossref]

J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
[Crossref] [PubMed]

2017 (2)

S. H. Chan, X. Wang, and O. A. Elgendy, “Plug-and-play ADMM for image restoration: Fixed-point convergence and applications,” IEEE Trans. Comput. Imag. 384–98 (2017).

J. Ma, S. Masoodian, D. A. Starkey, and E. R. Fossum, “Photon-number-resolving megapixel image sensor at room temperature without avalanche gain,” Optica 4, 1474–1481 (2017).
[Crossref]

2016 (4)

N. A. Dutton, I. Gyongy, L. Parmesan, and R. K. Henderson, “Single photon counting performance and noise analysis of CMOS SPAD-based image sensors,” Sensors 161122 (2016).
[Crossref] [PubMed]

S. H. Chan, O. A. Elgendy, and X. Wang, “Images from bits: Non-iterative image reconstruction for quanta image sensors,” Sensors 16, 1961 (2016).
[Crossref] [PubMed]

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

L. Azzari and A. Foi, “Variance stabilization for noisy+ estimate combination in iterative Poisson denoising,” IEEE Signal Process. Lett. 23, 1086–1090 (2016).
[Crossref]

2015 (2)

J. Ma and E. R. Fossum, “Quanta image sensor jot with sub 0.3 e-rms read noise and photon counting capability,” IEEE Electron Device Lett. 36, 926–928 (2015).
[Crossref]

J. Ma and E. R. Fossum, “A pump-gate jot device with high conversion gain for a quanta image sensor,” IEEE J. Electron Devices Soc. 3, 73–77 (2015).

2014 (1)

J. T. Korneliussen and K. Hirakawa, “Camera processing with chromatic aberration,” IEEE Trans. on Image Process. 23, 4539– 4552 (2014).
[Crossref]

2013 (1)

G. Jeon and E. Dubois, “Demosaicking of noisy Bayer-sampled color images with least-squares Luma-Chroma demultiplexing and noise level estimation,” IEEE Trans. Image Process. 22, 146–156 (2013).

2012 (1)

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “Bits from photons: Oversampled image acquisition using binary Poisson statistics,” IEEE Trans. Image Process. 21, 1421–1436 (2012).

2011 (1)

M. Makitalo and A. Foi, “Optimal inversion of the Anscombe transformation in low-count Poisson image denoising,” IEEE Trans. Image Process.,  20, 99–109 (2011).
[Crossref]

2008 (1)

E. Charbon, “Towards large scale CMOS single-photon detector arrays for lab-on-chip applications,” J. Phys. D: Appl. Phys. 41, 094010 (2008).
[Crossref]

2002 (1)

B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).

2001 (1)

J. Hynecek, “Impactron-a new solid state image intensifier,” IEEE Trans. Electron Devices 48, 2238–2241 (2001).
[Crossref]

1948 (1)

F. J. Anscombe, “The transformation of Poisson, binomial and negative-binomial data,” Biometrika 35, 246–254 (1948).
[Crossref]

Altmann, Y.

X. Ren, Y. Altmann, R. Tobin, A. Mccarthy, S. Mclaughlin, and G. S. Buller, “Wavelength-time coding for multispectral 3d imaging using single-photon lidar,” Opt.Express 26, 30146–30161 (2018).

Anscombe, F. J.

F. J. Anscombe, “The transformation of Poisson, binomial and negative-binomial data,” Biometrika 35, 246–254 (1948).
[Crossref]

Antolovic, I. M.

C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).

Ardelean, A.

J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
[Crossref] [PubMed]

Aull, B. F.

B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).

Azzari, L.

L. Azzari and A. Foi, “Variance stabilization for noisy+ estimate combination in iterative Poisson denoising,” IEEE Signal Process. Lett. 23, 1086–1090 (2016).
[Crossref]

L. Azzari and A. Foi, “Variance stabilization in Poisson image deblurring,” in IEEE International Symposium on Biomedical Imaging, (IEEE, 2017), 728–731.

Bronstein, A.

T. Remez, O. Litany, and A. Bronstein, “A picture is worth a billion bits: Real-time image reconstruction from dense binary threshold pixels,” in Proceedings of IEEE International Conference on Computational Photography, (IEEE, 2016), 1–9.

Bruschini, C.

J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
[Crossref] [PubMed]

C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).

P. Chandramouli, S. Burri, C. Bruschini, E. Charbon, and A. Kolb, “A little bit too much? High speed imaging from sparse photon counts,” arXiv preprint arXiv:1811.02396 (2018).

Buchholz, J.

J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
[Crossref] [PubMed]

Buller, G. S.

X. Ren, Y. Altmann, R. Tobin, A. Mccarthy, S. Mclaughlin, and G. S. Buller, “Wavelength-time coding for multispectral 3d imaging using single-photon lidar,” Opt.Express 26, 30146–30161 (2018).

Burri, S.

J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
[Crossref] [PubMed]

C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).

P. Chandramouli, S. Burri, C. Bruschini, E. Charbon, and A. Kolb, “A little bit too much? High speed imaging from sparse photon counts,” arXiv preprint arXiv:1811.02396 (2018).

Calder, N.

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

Chan, S. H.

O. A. Elgendy and S. H. Chan, “Optimal threshold design for quanta image sensor,” IEEE Trans. Comput. Imag. 4, 99–111 (2018).

S. H. Chan, X. Wang, and O. A. Elgendy, “Plug-and-play ADMM for image restoration: Fixed-point convergence and applications,” IEEE Trans. Comput. Imag. 384–98 (2017).

S. H. Chan, O. A. Elgendy, and X. Wang, “Images from bits: Non-iterative image reconstruction for quanta image sensors,” Sensors 16, 1961 (2016).
[Crossref] [PubMed]

S. H. Chan and Y. M. Lu, “Efficient image reconstruction for gigapixel quantum image sensors,” in Proceedings of IEEE Global Conference on Signal and Information Processing, (IEEE, 2014), 312–316.

J. H. Choi, O. A. Elgendy, and S. H. Chan, “Image reconstruction for quanta image sensors using deep neural networks,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (IEEE, 2018), 6543–6547.

S. H. Chan, “Performance analysis of plug-and-play admm: A graph signal processing perspective,” IEEE Trans. Comput. Imag. (2019).
[Crossref]

Chandramouli, P.

P. Chandramouli, S. Burri, C. Bruschini, E. Charbon, and A. Kolb, “A little bit too much? High speed imaging from sparse photon counts,” arXiv preprint arXiv:1811.02396 (2018).

Charbon, E.

J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
[Crossref] [PubMed]

E. Charbon, “Towards large scale CMOS single-photon detector arrays for lab-on-chip applications,” J. Phys. D: Appl. Phys. 41, 094010 (2008).
[Crossref]

E. Charbon, “Will avalanche photodiode arrays ever reach 1 megapixel,” in Proceedings of International Image Sensor Workshop, (IISS, 2007), 246–249.

C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).

P. Chandramouli, S. Burri, C. Bruschini, E. Charbon, and A. Kolb, “A little bit too much? High speed imaging from sparse photon counts,” arXiv preprint arXiv:1811.02396 (2018).

Choi, J. H.

J. H. Choi, O. A. Elgendy, and S. H. Chan, “Image reconstruction for quanta image sensors using deep neural networks,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (IEEE, 2018), 6543–6547.

Daniels, P. J.

B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).

Dubois, E.

G. Jeon and E. Dubois, “Demosaicking of noisy Bayer-sampled color images with least-squares Luma-Chroma demultiplexing and noise level estimation,” IEEE Trans. Image Process. 22, 146–156 (2013).

Dutton, N. A.

I. Gyongy, N. A. Dutton, and R. K. Henderson, “Single-photon tracking for high-speed vision,” Sensors 18323 (2018).
[Crossref]

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

N. A. Dutton, I. Gyongy, L. Parmesan, and R. K. Henderson, “Single photon counting performance and noise analysis of CMOS SPAD-based image sensors,” Sensors 161122 (2016).
[Crossref] [PubMed]

Dutton, N. A. W.

N. A. W. Dutton, L. Parmesan, A. J. Holmes, L. A. Grant, and R. K. Henderson, “320×240 oversampled digital single photon counting image sensor,” in Proceedings of IEEE 2014 Symposium on VLSI Circuits Digest of Technical Papers, (IEEE, 2014).

Elgendy, O. A.

O. A. Elgendy and S. H. Chan, “Optimal threshold design for quanta image sensor,” IEEE Trans. Comput. Imag. 4, 99–111 (2018).

S. H. Chan, X. Wang, and O. A. Elgendy, “Plug-and-play ADMM for image restoration: Fixed-point convergence and applications,” IEEE Trans. Comput. Imag. 384–98 (2017).

S. H. Chan, O. A. Elgendy, and X. Wang, “Images from bits: Non-iterative image reconstruction for quanta image sensors,” Sensors 16, 1961 (2016).
[Crossref] [PubMed]

J. H. Choi, O. A. Elgendy, and S. H. Chan, “Image reconstruction for quanta image sensors using deep neural networks,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (IEEE, 2018), 6543–6547.

Felton, B. J.

B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).

Foi, A.

L. Azzari and A. Foi, “Variance stabilization for noisy+ estimate combination in iterative Poisson denoising,” IEEE Signal Process. Lett. 23, 1086–1090 (2016).
[Crossref]

M. Makitalo and A. Foi, “Optimal inversion of the Anscombe transformation in low-count Poisson image denoising,” IEEE Trans. Image Process.,  20, 99–109 (2011).
[Crossref]

A. Foi, “Noise estimation and removal in MR imaging: The variance-stabilization approach,” in Proceedings of International symposium on biomedical imaging: from nano to macro, (IEEE, 2011), 1809–1814.

L. Azzari and A. Foi, “Variance stabilization in Poisson image deblurring,” in IEEE International Symposium on Biomedical Imaging, (IEEE, 2017), 728–731.

Fossum, E. R.

J. Ma, S. Masoodian, D. A. Starkey, and E. R. Fossum, “Photon-number-resolving megapixel image sensor at room temperature without avalanche gain,” Optica 4, 1474–1481 (2017).
[Crossref]

J. Ma and E. R. Fossum, “A pump-gate jot device with high conversion gain for a quanta image sensor,” IEEE J. Electron Devices Soc. 3, 73–77 (2015).

J. Ma and E. R. Fossum, “Quanta image sensor jot with sub 0.3 e-rms read noise and photon counting capability,” IEEE Electron Device Lett. 36, 926–928 (2015).
[Crossref]

S. Masoodian, J. Ma, D. Starkey, Y. Yamashita, and E. R. Fossum, “A 1Mjot 1040fps 0.22 e-rms stacked BSI quanta image sensor with Cluster-Parallel Readout,” in Proceedings of the International Image Sensor Workshop, vol. 30 (IISS, 2017), 230–233.

E. R. Fossum, “Some Thoughts on Future Digital Still Cameras,” in Image Sensors and Signal Processing for Digital Still Cameras, (CRC, 2006), 305–314.

E. R. Fossum, “Gigapixel digital film sensor (DFS) proposal,” in Nanospace Manipulation of Photons and Electrons for Nanovision Systems, The 7th Takayanagi Kenjiro Memorial Symposium and the 2nd International Symposium on Nanovision Science, Hamamatsu, Japan, (2005).

E. R. Fossum, “Active pixel sensors: Are CCDs dinosaurs?” in Charge-Coupled Devices and Solid State Optical Sensors III, vol. 1900 (International Society for Optics and Photonics, 1993), 2–15.

Gnecchi, S.

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

Grant, L. A.

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

N. A. W. Dutton, L. Parmesan, A. J. Holmes, L. A. Grant, and R. K. Henderson, “320×240 oversampled digital single photon counting image sensor,” in Proceedings of IEEE 2014 Symposium on VLSI Circuits Digest of Technical Papers, (IEEE, 2014).

Gyongy, I.

I. Gyongy, N. A. Dutton, and R. K. Henderson, “Single-photon tracking for high-speed vision,” Sensors 18323 (2018).
[Crossref]

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

N. A. Dutton, I. Gyongy, L. Parmesan, and R. K. Henderson, “Single photon counting performance and noise analysis of CMOS SPAD-based image sensors,” Sensors 161122 (2016).
[Crossref] [PubMed]

Heinrichs, R. M.

B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).

Henderson, R. K.

I. Gyongy, N. A. Dutton, and R. K. Henderson, “Single-photon tracking for high-speed vision,” Sensors 18323 (2018).
[Crossref]

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

N. A. Dutton, I. Gyongy, L. Parmesan, and R. K. Henderson, “Single photon counting performance and noise analysis of CMOS SPAD-based image sensors,” Sensors 161122 (2016).
[Crossref] [PubMed]

N. A. W. Dutton, L. Parmesan, A. J. Holmes, L. A. Grant, and R. K. Henderson, “320×240 oversampled digital single photon counting image sensor,” in Proceedings of IEEE 2014 Symposium on VLSI Circuits Digest of Technical Papers, (IEEE, 2014).

Hirakawa, K.

J. T. Korneliussen and K. Hirakawa, “Camera processing with chromatic aberration,” IEEE Trans. on Image Process. 23, 4539– 4552 (2014).
[Crossref]

K. Hirakawa, X. Meng, and P. J. Wolfe, “A framework for wavelet-based analysis and processing of color filter array images with applications to denoising and demosaicing,” in IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1 (IEEE, 2007), 597–600.

Holmes, A. J.

N. A. W. Dutton, L. Parmesan, A. J. Holmes, L. A. Grant, and R. K. Henderson, “320×240 oversampled digital single photon counting image sensor,” in Proceedings of IEEE 2014 Symposium on VLSI Circuits Digest of Technical Papers, (IEEE, 2014).

Hynecek, J.

J. Hynecek, “Impactron-a new solid state image intensifier,” IEEE Trans. Electron Devices 48, 2238–2241 (2001).
[Crossref]

Jeon, G.

G. Jeon and E. Dubois, “Demosaicking of noisy Bayer-sampled color images with least-squares Luma-Chroma demultiplexing and noise level estimation,” IEEE Trans. Image Process. 22, 146–156 (2013).

Kolb, A.

P. Chandramouli, S. Burri, C. Bruschini, E. Charbon, and A. Kolb, “A little bit too much? High speed imaging from sparse photon counts,” arXiv preprint arXiv:1811.02396 (2018).

Korneliussen, J. T.

J. T. Korneliussen and K. Hirakawa, “Camera processing with chromatic aberration,” IEEE Trans. on Image Process. 23, 4539– 4552 (2014).
[Crossref]

Krieger, J.

J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
[Crossref] [PubMed]

Landers, D. J.

B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).

Langowski, J.

J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
[Crossref] [PubMed]

Lindner, S.

C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).

Litany, O.

T. Remez, O. Litany, and A. Bronstein, “A picture is worth a billion bits: Real-time image reconstruction from dense binary threshold pixels,” in Proceedings of IEEE International Conference on Computational Photography, (IEEE, 2016), 1–9.

Loomis, A. H.

B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).

Lu, Y. M.

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “Bits from photons: Oversampled image acquisition using binary Poisson statistics,” IEEE Trans. Image Process. 21, 1421–1436 (2012).

S. H. Chan and Y. M. Lu, “Efficient image reconstruction for gigapixel quantum image sensors,” in Proceedings of IEEE Global Conference on Signal and Information Processing, (IEEE, 2014), 312–316.

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “An optimal algorithm for reconstructing images from binary measurements,” in Computational Imaging VIII, vol. 7533 (International Society for Optics and Photonics, 2010), 75330K.
[Crossref]

R. A. Rojas, W. Luo, V. Murray, and Y. M. Lu, “Learning optimal parameters for binary sensing image reconstruction algorithms,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2017), 2791–2795.

Luo, W.

R. A. Rojas, W. Luo, V. Murray, and Y. M. Lu, “Learning optimal parameters for binary sensing image reconstruction algorithms,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2017), 2791–2795.

Ma, J.

J. Ma, S. Masoodian, D. A. Starkey, and E. R. Fossum, “Photon-number-resolving megapixel image sensor at room temperature without avalanche gain,” Optica 4, 1474–1481 (2017).
[Crossref]

J. Ma and E. R. Fossum, “A pump-gate jot device with high conversion gain for a quanta image sensor,” IEEE J. Electron Devices Soc. 3, 73–77 (2015).

J. Ma and E. R. Fossum, “Quanta image sensor jot with sub 0.3 e-rms read noise and photon counting capability,” IEEE Electron Device Lett. 36, 926–928 (2015).
[Crossref]

S. Masoodian, J. Ma, D. Starkey, Y. Yamashita, and E. R. Fossum, “A 1Mjot 1040fps 0.22 e-rms stacked BSI quanta image sensor with Cluster-Parallel Readout,” in Proceedings of the International Image Sensor Workshop, vol. 30 (IISS, 2017), 230–233.

Makitalo, M.

M. Makitalo and A. Foi, “Optimal inversion of the Anscombe transformation in low-count Poisson image denoising,” IEEE Trans. Image Process.,  20, 99–109 (2011).
[Crossref]

Masoodian, S.

J. Ma, S. Masoodian, D. A. Starkey, and E. R. Fossum, “Photon-number-resolving megapixel image sensor at room temperature without avalanche gain,” Optica 4, 1474–1481 (2017).
[Crossref]

S. Masoodian, J. Ma, D. Starkey, Y. Yamashita, and E. R. Fossum, “A 1Mjot 1040fps 0.22 e-rms stacked BSI quanta image sensor with Cluster-Parallel Readout,” in Proceedings of the International Image Sensor Workshop, vol. 30 (IISS, 2017), 230–233.

Mccarthy, A.

X. Ren, Y. Altmann, R. Tobin, A. Mccarthy, S. Mclaughlin, and G. S. Buller, “Wavelength-time coding for multispectral 3d imaging using single-photon lidar,” Opt.Express 26, 30146–30161 (2018).

Mclaughlin, S.

X. Ren, Y. Altmann, R. Tobin, A. Mccarthy, S. Mclaughlin, and G. S. Buller, “Wavelength-time coding for multispectral 3d imaging using single-photon lidar,” Opt.Express 26, 30146–30161 (2018).

Meng, X.

K. Hirakawa, X. Meng, and P. J. Wolfe, “A framework for wavelet-based analysis and processing of color filter array images with applications to denoising and demosaicing,” in IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1 (IEEE, 2007), 597–600.

Murray, V.

R. A. Rojas, W. Luo, V. Murray, and Y. M. Lu, “Learning optimal parameters for binary sensing image reconstruction algorithms,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2017), 2791–2795.

Parmesan, L.

N. A. Dutton, I. Gyongy, L. Parmesan, and R. K. Henderson, “Single photon counting performance and noise analysis of CMOS SPAD-based image sensors,” Sensors 161122 (2016).
[Crossref] [PubMed]

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

N. A. W. Dutton, L. Parmesan, A. J. Holmes, L. A. Grant, and R. K. Henderson, “320×240 oversampled digital single photon counting image sensor,” in Proceedings of IEEE 2014 Symposium on VLSI Circuits Digest of Technical Papers, (IEEE, 2014).

Pellegrini, S.

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

Rae, B. R.

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

Remez, T.

T. Remez, O. Litany, and A. Bronstein, “A picture is worth a billion bits: Real-time image reconstruction from dense binary threshold pixels,” in Proceedings of IEEE International Conference on Computational Photography, (IEEE, 2016), 1–9.

Ren, X.

X. Ren, Y. Altmann, R. Tobin, A. Mccarthy, S. Mclaughlin, and G. S. Buller, “Wavelength-time coding for multispectral 3d imaging using single-photon lidar,” Opt.Express 26, 30146–30161 (2018).

Rojas, R. A.

R. A. Rojas, W. Luo, V. Murray, and Y. M. Lu, “Learning optimal parameters for binary sensing image reconstruction algorithms,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2017), 2791–2795.

Sbaiz, L.

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “Bits from photons: Oversampled image acquisition using binary Poisson statistics,” IEEE Trans. Image Process. 21, 1421–1436 (2012).

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “An optimal algorithm for reconstructing images from binary measurements,” in Computational Imaging VIII, vol. 7533 (International Society for Optics and Photonics, 2010), 75330K.
[Crossref]

Starkey, D.

S. Masoodian, J. Ma, D. Starkey, Y. Yamashita, and E. R. Fossum, “A 1Mjot 1040fps 0.22 e-rms stacked BSI quanta image sensor with Cluster-Parallel Readout,” in Proceedings of the International Image Sensor Workshop, vol. 30 (IISS, 2017), 230–233.

Starkey, D. A.

Tobin, R.

X. Ren, Y. Altmann, R. Tobin, A. Mccarthy, S. Mclaughlin, and G. S. Buller, “Wavelength-time coding for multispectral 3d imaging using single-photon lidar,” Opt.Express 26, 30146–30161 (2018).

Ulku, A. C.

C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).

Vetterli, M.

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “Bits from photons: Oversampled image acquisition using binary Poisson statistics,” IEEE Trans. Image Process. 21, 1421–1436 (2012).

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “An optimal algorithm for reconstructing images from binary measurements,” in Computational Imaging VIII, vol. 7533 (International Society for Optics and Photonics, 2010), 75330K.
[Crossref]

Wang, X.

S. H. Chan, X. Wang, and O. A. Elgendy, “Plug-and-play ADMM for image restoration: Fixed-point convergence and applications,” IEEE Trans. Comput. Imag. 384–98 (2017).

S. H. Chan, O. A. Elgendy, and X. Wang, “Images from bits: Non-iterative image reconstruction for quanta image sensors,” Sensors 16, 1961 (2016).
[Crossref] [PubMed]

Wolf, M.

C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).

Wolfe, P. J.

K. Hirakawa, X. Meng, and P. J. Wolfe, “A framework for wavelet-based analysis and processing of color filter array images with applications to denoising and demosaicing,” in IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1 (IEEE, 2007), 597–600.

Yamashita, Y.

S. Masoodian, J. Ma, D. Starkey, Y. Yamashita, and E. R. Fossum, “A 1Mjot 1040fps 0.22 e-rms stacked BSI quanta image sensor with Cluster-Parallel Readout,” in Proceedings of the International Image Sensor Workshop, vol. 30 (IISS, 2017), 230–233.

Yang, F.

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “Bits from photons: Oversampled image acquisition using binary Poisson statistics,” IEEE Trans. Image Process. 21, 1421–1436 (2012).

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “An optimal algorithm for reconstructing images from binary measurements,” in Computational Imaging VIII, vol. 7533 (International Society for Optics and Photonics, 2010), 75330K.
[Crossref]

Young, D. J.

B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).

Zhang, C.

C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).

Biometrika (1)

F. J. Anscombe, “The transformation of Poisson, binomial and negative-binomial data,” Biometrika 35, 246–254 (1948).
[Crossref]

Biophys. J. (1)

J. Buchholz, J. Krieger, C. Bruschini, S. Burri, A. Ardelean, E. Charbon, and J. Langowski, “Widefield high frame rate single-photon SPAD imagers for SPIM-FCS,” Biophys. J. 114, 2455–2464 (2018).
[Crossref] [PubMed]

IEEE Electron Device Lett. (1)

J. Ma and E. R. Fossum, “Quanta image sensor jot with sub 0.3 e-rms read noise and photon counting capability,” IEEE Electron Device Lett. 36, 926–928 (2015).
[Crossref]

IEEE J. Electron Devices Soc. (1)

J. Ma and E. R. Fossum, “A pump-gate jot device with high conversion gain for a quanta image sensor,” IEEE J. Electron Devices Soc. 3, 73–77 (2015).

IEEE Signal Process. Lett. (1)

L. Azzari and A. Foi, “Variance stabilization for noisy+ estimate combination in iterative Poisson denoising,” IEEE Signal Process. Lett. 23, 1086–1090 (2016).
[Crossref]

IEEE Trans. Comput. Imag. (2)

S. H. Chan, X. Wang, and O. A. Elgendy, “Plug-and-play ADMM for image restoration: Fixed-point convergence and applications,” IEEE Trans. Comput. Imag. 384–98 (2017).

O. A. Elgendy and S. H. Chan, “Optimal threshold design for quanta image sensor,” IEEE Trans. Comput. Imag. 4, 99–111 (2018).

IEEE Trans. Electron Devices (2)

J. Hynecek, “Impactron-a new solid state image intensifier,” IEEE Trans. Electron Devices 48, 2238–2241 (2001).
[Crossref]

N. A. Dutton, I. Gyongy, L. Parmesan, S. Gnecchi, N. Calder, B. R. Rae, S. Pellegrini, L. A. Grant, and R. K. Henderson, “A SPAD-based QVGA image sensor for single-photon counting and quanta imaging,” IEEE Trans. Electron Devices 63, 189–196 (2016).
[Crossref]

IEEE Trans. Image Process. (3)

M. Makitalo and A. Foi, “Optimal inversion of the Anscombe transformation in low-count Poisson image denoising,” IEEE Trans. Image Process.,  20, 99–109 (2011).
[Crossref]

G. Jeon and E. Dubois, “Demosaicking of noisy Bayer-sampled color images with least-squares Luma-Chroma demultiplexing and noise level estimation,” IEEE Trans. Image Process. 22, 146–156 (2013).

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “Bits from photons: Oversampled image acquisition using binary Poisson statistics,” IEEE Trans. Image Process. 21, 1421–1436 (2012).

IEEE Trans. on Image Process. (1)

J. T. Korneliussen and K. Hirakawa, “Camera processing with chromatic aberration,” IEEE Trans. on Image Process. 23, 4539– 4552 (2014).
[Crossref]

J. Phys. D: Appl. Phys. (1)

E. Charbon, “Towards large scale CMOS single-photon detector arrays for lab-on-chip applications,” J. Phys. D: Appl. Phys. 41, 094010 (2008).
[Crossref]

Linc. Lab. J. (1)

B. F. Aull, A. H. Loomis, D. J. Young, R. M. Heinrichs, B. J. Felton, P. J. Daniels, and D. J. Landers, “Geiger-mode avalanche photodiodes for three-dimensional imaging,” Linc. Lab. J. 13, 335–349 (2002).

Opt.Express (1)

X. Ren, Y. Altmann, R. Tobin, A. Mccarthy, S. Mclaughlin, and G. S. Buller, “Wavelength-time coding for multispectral 3d imaging using single-photon lidar,” Opt.Express 26, 30146–30161 (2018).

Optica (1)

Sensors (3)

N. A. Dutton, I. Gyongy, L. Parmesan, and R. K. Henderson, “Single photon counting performance and noise analysis of CMOS SPAD-based image sensors,” Sensors 161122 (2016).
[Crossref] [PubMed]

I. Gyongy, N. A. Dutton, and R. K. Henderson, “Single-photon tracking for high-speed vision,” Sensors 18323 (2018).
[Crossref]

S. H. Chan, O. A. Elgendy, and X. Wang, “Images from bits: Non-iterative image reconstruction for quanta image sensors,” Sensors 16, 1961 (2016).
[Crossref] [PubMed]

Other (18)

E. Charbon, “Will avalanche photodiode arrays ever reach 1 megapixel,” in Proceedings of International Image Sensor Workshop, (IISS, 2007), 246–249.

E. R. Fossum, “Some Thoughts on Future Digital Still Cameras,” in Image Sensors and Signal Processing for Digital Still Cameras, (CRC, 2006), 305–314.

E. R. Fossum, “Gigapixel digital film sensor (DFS) proposal,” in Nanospace Manipulation of Photons and Electrons for Nanovision Systems, The 7th Takayanagi Kenjiro Memorial Symposium and the 2nd International Symposium on Nanovision Science, Hamamatsu, Japan, (2005).

E. R. Fossum, “Active pixel sensors: Are CCDs dinosaurs?” in Charge-Coupled Devices and Solid State Optical Sensors III, vol. 1900 (International Society for Optics and Photonics, 1993), 2–15.

N. A. W. Dutton, L. Parmesan, A. J. Holmes, L. A. Grant, and R. K. Henderson, “320×240 oversampled digital single photon counting image sensor,” in Proceedings of IEEE 2014 Symposium on VLSI Circuits Digest of Technical Papers, (IEEE, 2014).

C. Bruschini, S. Burri, S. Lindner, A. C. Ulku, C. Zhang, I. M. Antolovic, M. Wolf, and E. Charbon, “Monolithic SPAD arrays for high-performance, time-resolved single-photon imaging,” in Proceedings of International Conference on Optical MEMS and Nanophotonics, (IEEE, 2018).

S. H. Chan and Y. M. Lu, “Efficient image reconstruction for gigapixel quantum image sensors,” in Proceedings of IEEE Global Conference on Signal and Information Processing, (IEEE, 2014), 312–316.

K. Hirakawa, X. Meng, and P. J. Wolfe, “A framework for wavelet-based analysis and processing of color filter array images with applications to denoising and demosaicing,” in IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1 (IEEE, 2007), 597–600.

S. H. Chan, “Performance analysis of plug-and-play admm: A graph signal processing perspective,” IEEE Trans. Comput. Imag. (2019).
[Crossref]

A. Foi, “Noise estimation and removal in MR imaging: The variance-stabilization approach,” in Proceedings of International symposium on biomedical imaging: from nano to macro, (IEEE, 2011), 1809–1814.

S. Masoodian, J. Ma, D. Starkey, Y. Yamashita, and E. R. Fossum, “A 1Mjot 1040fps 0.22 e-rms stacked BSI quanta image sensor with Cluster-Parallel Readout,” in Proceedings of the International Image Sensor Workshop, vol. 30 (IISS, 2017), 230–233.

“Zylus 4.2 PLUS,” https://andor.oxinst.com/products/scmos-camera-series/zyla-4-2-scmos . Accessed: 2019-04-21.

F. Yang, Y. M. Lu, L. Sbaiz, and M. Vetterli, “An optimal algorithm for reconstructing images from binary measurements,” in Computational Imaging VIII, vol. 7533 (International Society for Optics and Photonics, 2010), 75330K.
[Crossref]

T. Remez, O. Litany, and A. Bronstein, “A picture is worth a billion bits: Real-time image reconstruction from dense binary threshold pixels,” in Proceedings of IEEE International Conference on Computational Photography, (IEEE, 2016), 1–9.

J. H. Choi, O. A. Elgendy, and S. H. Chan, “Image reconstruction for quanta image sensors using deep neural networks,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (IEEE, 2018), 6543–6547.

P. Chandramouli, S. Burri, C. Bruschini, E. Charbon, and A. Kolb, “A little bit too much? High speed imaging from sparse photon counts,” arXiv preprint arXiv:1811.02396 (2018).

R. A. Rojas, W. Luo, V. Murray, and Y. M. Lu, “Learning optimal parameters for binary sensing image reconstruction algorithms,” in Proceedings of IEEE International Conference on Image Processing, (IEEE, 2017), 2791–2795.

L. Azzari and A. Foi, “Variance stabilization in Poisson image deblurring,” in IEEE International Symposium on Biomedical Imaging, (IEEE, 2017), 728–731.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1 (a) One 1-bit frame. (b) Reconstructed color image using 50 frames of 1-bit input with threshold q = 4. (c) One 5-bit frame. (d) Reconstructed color image using 10 frames of 5-bit input. The average number ofphotons per frame is 5.
Fig. 2
Fig. 2 Quanta Image Sensor (QIS) vs. CMOS Image Sensor (CIS). The top row shows a simulated CIS data at a photon level same as the QIS. The second row shows the real analog sensor data obtained from our prototype camera. For each photon flux level, we show both the raw input data and the denoised data. The CIS is assumed to have a read noise of 1.2 e r.m.s., and we do not considered dark count. With dark count, the performance of CIS will deteriorate even more.
Fig. 3
Fig. 3 QIS Imaging Model. When the scene image arrives at the sensor, the color filter array first selects the wavelength according to the colors. Each color pixel is then sensed using a photon-detector. In single-bit mode the detector reports a binary value, and in multi-bit mode the detector reports an integer up to the saturation limit. The measured data contains three subsampled sequences, each representing a measurement in the color channel.
Fig. 4
Fig. 4 Gigajot prototype QIS camera module and the QIS sensor chip. Note that there is no additional optics required besides a simple focusing lens. The camera is powered by standard 5V DC input, and has a USB3 data interface to transmit data to external storage.
Fig. 5
Fig. 5 Measured photo-electron count from a real sensor. The non-integer values of the count are due to the read noise of the readout circuit. However, with the ultra-low read noise provided by the QIS device, its negative effect on the photon-countingis negligible by setting appropriate thresholds.
Fig. 6
Fig. 6 The image reconstruction pipeline consists of (i) a temporal binning step to sum the input frames, (ii) a variance stabilizing transform T to transform the measurement so that the variance is stabilized, (iii) a joint reconstruction and demosaicing algorithm to recovery the color, (iv) an inverse transform to compensate the forward transform, and (v) a tone mapping operation to correct the contrast.
Fig. 7
Fig. 7 In the future when QIS can achieve higher spatial resolution, we can use four color jots to reconstruct one pixel. In this case, we can bypass the iterative ADMM algorithm and use a one-shot denoising method.
Fig. 8
Fig. 8 Simulated QIS experiment. The goal of this experiment is to compare the proposed iterative algorithm with existing methods. We assume the observed Bayer RGB image is from a 3-bit QIS sensor. (a) Ground Truth; (b) One 3-bit QIS frame; (c) MATLAB demosaic preceded and followed by BM3D; (d) LSLCD [36]; (e) Hirakawa’s PSDD method [38], with a built-in wavelet shrinkage denoiser; (f) Proposed method with BM3D.
Fig. 9
Fig. 9 Synthetic experiment for quantitative evaluation. [Top row]: One frame of the QIS measurements using different number of bits. [Bottom row]: Reconstructed images using the proposed method with 20 frames of QIS data. The average photon counts per pixel are 0.25, 0.75, 1.75 and 3.75 for 1-bit, 2-bit, 3-bit and 4-bit QIS, respectively.
Fig. 10
Fig. 10 Real QIS image reconstruction. The exposure time for each frame is 50 μs. The average number of photons per frame is 4.2, 3.0, 1.9, and 2.9 for each image respectively. Both methods use 4 frames for reconstruction. The raw data has a resolution of 1024 × 1024 pixels. The ADMM method retains the resolution, whereas the non-iterative method reduces the resolution to 512 × 512. Reconstruction using both the methods are shown at the same size for easier visual comparison. Notice that the non-iterative algorithm is able to achieve a visual quality almost similar to the ADMM method.

Tables (1)

Tables Icon

Table 1 Comparison of the available image sensor technologies.a

Equations (15)

Equations on this page are rendered with MathJax. Learn more.

v = [ v r ; v g ; v b ] = α Gc 3 M ,
θ = S r v r + S g v g + S b v b M .
θ = α SGc .
( Y = y ) = m = 1 M θ m y m e θ m y m ! .
B m = { Y m , Y m < 2 L 1 , 2 L 1 , Y m 2 L 1.
( Y = y ) = t = 1 T m = 1 M θ m y m , t e θ m y m , t ! ,
Z m = t = 1 T B m , t .
[ Z m = k | θ m ] = ( T k ) Ψ q ( θ m ) 1 k ( 1 Ψ q ( θ m ) ) k ,
[ Z m = k | θ m ] = ( T θ m ) k e T θ m k ! .
θ = argmax θ   m = 1 M [ Z m = z m | θ m ] = def M ( z ) = { Ψ q 1 ( 1 z / T ) , Single bit , z / T , Multi bit .
β = T ( z ) = def { T + 1 2 sin 1 z + 3 8 T + 3 4 , Single bit , z + 3 8 , Multi bit .
T ( z ) = T ( S z ˜ ) = S T ( z ˜ ) .
v ^ = argmin x   Sx T ( z ) 2 + λ g ( x ) ,
x ( k + 1 ) = ( S T S + ρ I ) 1 ( S T T ( z ) + ρ ( v ( k ) u ( k ) ) ) ,
v ( k + 1 ) = D ρ / λ ( x ( k + 1 ) + u ( k ) ) ,

Metrics