Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

LinoSPAD2: an FPGA-based, hardware-reconfigurable 512×1 single-photon camera system

Open Access Open Access

Abstract

We report on LinoSPAD2, a single-photon camera system, comprising a 512×1 single-photon avalanche diode (SPAD) front-end and one or two FPGA-based back-ends. Digital signals generated by the SPADs are processed by the FPGA in real time, whereas the FPGA offers full reconfigurability at a very high level of granularity both in time and space domains. The LinoSPAD2 camera system can process 512 SPADs simultaneously through 256 channels, duplicated on each FPGA-based back-end, with a bank of 64 time-to-digital converters (TDCs) operating at 133 MSa/s, whereas each TDC has a time resolution of 20 ps (LSB). To the best of our knowledge, LinoSPAD2 is the first fully reconfigurable SPAD camera system of large format. The SPAD front-end features a pitch of 26.2 μm, a native fill factor of 25.1%, and a microlens array achieving 2.3× concentration factor. At room temperature, the median dark count rate (DCR) is 80 cps at 7 V excess bias, the peak photon detection probability (PDP) is 53% at 520 nm wavelength, and the single-photon timing resolution (SPTR) is 50 ps FWHM. The instrument response function (IRF) is around 100 ps FWHM at system level. The LinoSPAD2 camera system is suitable for numerous applications, including LiDAR imaging, heralded spectroscopy, compressive Raman sensing, and other computational imaging techniques.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Single-photon avalanche diodes (SPADs), or Geiger-mode avalanche photodiodes (GM-APDs), are single-photon detectors capable of detecting single photon events and to generate a digital pulse with a certain probability, known as photon detection probability (PDP). Detected photons can be counted individually or associated with a time stamp characterizing their time-of-arrival with picosecond precision [1]. SPADs are suitable for fast gating, which is implemented as an electronic shutter with less than 100 ps raise and fall time and a width as low as a few hundred picoseconds [2].

Thanks to their CMOS compatibility [3], SPADs can be used in a variety of applications requiring time-resolved imaging [1,4], whereas a wide range of architectures has been explored, from single high-performance pixels to Megapixel imagers with gating and/or timing electronics at pixel or column level [513]. In SPADs, a p-n or n-p junction biased above breakdown, i.e. in Geiger mode, must be coupled to a quenching and recharge circuitry to avoid distruction of the junction upon triggering of an avalanche. The same circuitry is also used to detect avalanches triggered by photon detection and thermal events and to convert them onto a digital signal, which can then be routed to a logic function. One-dimensional or linear arrays of SPADs, such as the one presented in this paper, allow to optimize fill factor by pushing pixel circuitry to the periphery of the chip.

Both time-correlated and uncorrelated applications have been explored over the years. In time-uncorrelated applications, large-format SPAD arrays operating at over 100 kfps, with time-gating have been proposed, where the natively digital nature of SPAD-based pixels enables one to manipulate their digital output directly without going through and A/D converter. This, combined with a virtually noiseless read-out, is useful when applied to computational imaging, as it enables high-speed movement correction at a single-photon level in large scenes for image quality optimization [14]. In time-correlated applications, one exploits high intrinsic timing precision of SPADs, which is generally characterized as single-photon timing resolution (SPTR), for instance, to ascertain the time-of-flight on a pixel basis. Other important time-correlated applications exist based on individual SPADs and SPAD arrays, such as fluorescence lifetime imaging microscopy (FLIM), near-infrared optical tomography, non-line-of-sight imaging, ghost imaging, and quantum random number generation, just to name a few.

All these applications share a common need of fast data acquisition and high-performance processing. Some of the commonly used techniques are photon counting, histogramming, single-photon synchronous detection [1517], which are generally integrated on chip along with the SPADs. Thus, only limited or partial programmability is possible. Examples include different histogram binning and on-chip data compression levels, often obtained by employing some degree of circuit sharing to minimize silicon real estate [8,10,11,16,1822]. Programmability by means of digital processing units combined to a backside-illuminated SPAD array has been reported in [12], in-pixel reconfigurable logic in [23], and modular photon processors in [9,24], allowing to switch between intensity and time-resolved operating modes.

This paper details the architecture and performance of the LinoSPAD2 single-photon camera system. To the best of our knowledge, LinoSPAD2 is the first fully-reconfigurable photon-counting camera system, where the SPAD array is directly coupled to massively parallelized processing logic implemented in two field-programmable gate-arrays (FPGAs). Thanks to direct access to light information with the granularity of a single photon and picosecond timing resolution, LinoSPAD2 enables one to implement unprecedented functionality on a very large number of pixels in a fashion that is effectively free of read-out noise. As an example of the fully reconfigurable FPGA architecture, the end-user might need a coarser time resolution for the target application with a better non-linearity: in this case the TDC binning is performed directly on the FPGA while the timestamp histogram is building up, without burdening the end user with this task. This has been achieved without degradation of the native SPAD performance and mitigating the typical issues of high performance, fully integrated designs, such as additional design costs, extended fabrication cycles and operating complexity. The LinoSPAD concept originally launched in [25,26] has been totally redesigned and in this paper we outline the result taking advantage of a state-of-the-art SPAD technology and enhanced firmware to allow the exploration of new applications [27].

2. Image sensor architecture

LinoSPAD2 features a linear array of 512${\times }$1 SPAD pixels implemented in a 0.18 $\mu$m CMOS process. A micrograph of the IC is shown in Fig. 1. Each pixel includes a SPAD based on the p-i-n configuration described in [28], a dedicated passive quenching and recharge circuitry, and an inverter for digital signal forming and impedance adaptation. Quenching and recharge are performed by a 600-k$\Omega$ resistor, while AC-coupling through an n-well capacitor to a diode is done to limit the voltage on the digital logic when large excess bias voltages are used. A transistor controlled by $V_{ctrl}$ is used to leak a very small current from the limiting diode, so as to avoid capacitor potential buildup at the input of the inverter. Thanks to this solution, high excess bias voltage can be achieved to optimize PDP and SPTR. The pixel schematic is shown in Fig. 2, where $V_{op}$, $V_{diode}$, and $V_{ctrl}$ are used to bias and control the pixels and $V_{dd}$ to power the logic circuits. $V_{op} = V_{BD} + V_{EX}$ has to be adjusted, so as to achieve the intended excess bias voltage $V_{EX}$ for a given breakdown voltage $V_{BD}$, which varies as a function of temperature. The novel aspect of this front-end design is to enable high photon count rates at the SPAD output, since the AC coupling circuit makes the anode signal derivative, leading to spiking outputs detected by the inverter. Even if the SPAD anode RC constant is low, the detected photons can thus be recorded even in presence of high illumination levels. This scheme is described in Section 2.1.

 figure: Fig. 1.

Fig. 1. LinoSPAD2 IC micrograph detail. Microlenses are imprinted on top of the SPADs, with the alignment crosses drawn with the final metal layer on the array side (one cross is visible on the left).

Download Full Size | PDF

 figure: Fig. 2.

Fig. 2. LinoSPAD2 pixel circuit with voltage waveforms highlighted, enabling individual SPAD readout and 1:1 coupling to external FPGAs. This configuration enables high excess bias voltages thanks to AC coupling to logic circuits and higher photon count rates. $V_{a}$ indicates the peak voltage of the inverter input, for which the maximum is set by $V_{diode}$.

Download Full Size | PDF

The output of each pixel is connected to a dedicated digital I/O pad, thus enabling fully parallel readout of the SPADs. Control and power supply pads are placed every 20 signal I/O blocks to minimize IR drop, especially when high photon fluxes may induce a localized reduction of $V_{op}$ and thus of excess bias. A daughterboard PCB hosts the sensor, which is mounted in a chip-on-board configuration. A motherboard is connected to the daughterboard via high-density multipole connectors. The motherboard hosts a Spartan 6 XC6SLX150 Xilinx FPGA capable of supporting up to 150k logic elements packed in 20k slices and other discrete components for power management. FPGA programming and data transmission to a host PCB are enabled by a Cypress FX3 USB3.0 transceiver chip. Since a Spartan 6 has 498 available I/Os, a single motherboard is not sufficient to read out the whole SPAD array. Two motherboards are therefore connected in parallel to a single daughterboard. Each motherboard, capable of processing 256 pixels, is based on [29], whereas the firmware and software are new. The USB connection enables continuous streaming of 200 Mbps and up to 3 Gbps to the host PC.

2.1 High-voltage compact monostable front-end

The pixel front-end design has been redesigned with the purpose of allowing a high excess bias voltage to be applied on the SPAD device while achieving a high maximum photon count rate. Several solutions to sustain a high excess bias on the SPAD device have been presented in the literature, such as introducing cascode transistors between the SPAD anode and the quenching device [3032]. However, the SPAD excess bias allowed on-chip is still limited due to the breakdown of the front-end MOS devices. The novel front-end design is shown in Fig. 2 together with the important waveforms occurring for two close photon events. With the proposed pixel scheme, the SPAD can reach higher excess biases, for which the authors tested values up to 8 V. The diode placed after the capacitor plate protects the inverter and M0 transistors from the anode voltage swing (0 to $V_{ex}$). Since the polysilicon resistor and the metal-on-metal (MoM) capacitor can sustain high voltages (up to tens of V) the pixel is able to reach a high excess bias. This is a key factor for applications in which the SPAD must have a high sensitivity. In addition to this feature, the MoM capacitor together with M0 are acting as a high-pass filter on the SPAD anode, which attenuates the low-frequency components of the voltage while reaching the inverter input. This transforms de facto the SPAD anode voltage into a high-frequency only spiking signal, modulated by the $V_{ctrl}$ voltage. With this scheme, even if the SPAD anode is recharging, as long as the SPAD is in breakdown regime it is able to detect another photon and the event is transmitted to the output.

We call this architecture a compact monostable circuit. The authors in [33,34] implemented an AC coupling in the SPAD front-end, but in a reversed pixel configuration and for another aim, i.e. to ensure DC compatibility of the capacitor output voltage with the digital CMOS logic implemented. In addition to that, the maximum applicable excess bias voltage was still limited by the anode swing, while in our scheme the diode limits this voltage, allowing a greater and configurable maximum anode voltage swing. Simulated values for this pixel reach up to 100 Mcps while the maximum measured countrate of the system is around 24 Mcps per SPAD, limited mainly by the FPGA readout, the PCB parasitic capacitance and the pixel direct readout to pad without further digital counting on-chip.

With these two novel features, i.e. very high SPAD excess bias voltages and high photon count rate detection, the LinoSPAD2 chip excels in applications where high sensitivity and/or high photon count rates are necessary.

2.2 System architecture

The LinoSPAD2 system is shown in Fig. 3. It features one or two custom-made motherboards and a daughterboard, which hosts the 512$\times$1 line sensor. Several read-out and processing topologies can be implemented in the FPGA and also modified in quasi real-time. They range from pixel-based digital counters to shared TDC arrays capable of on-the-fly histogram generation and calibration. Four representative examples are shown in Fig. 4, based on [26]. Fig. 4(a) illustrates the implementation of 256 parallel 32 bit counters, one for each SPAD output of a sensor half. Since not all applications do require picosecond timing resolution but rather high throughput, the TDC LSB can be increased or, in other words, bins can be merged to achieve nanosecond LSB. This operation, known as binning, has been achieved by operating the counters at 100 MHz, coupled to a double buffering memory scheme to avoid missing events, with integration times down to a few microseconds. If counting on a shorter timescale is required, gates can be implemented, as shown in Fig. 4(b), potentially employing multiple counters. The use of programmable delay elements within FPGAs makes it possible to instantiate gates with 100-ps precision.

 figure: Fig. 3.

Fig. 3. LinoSPAD2 system. One of the two custom-made motherboards, each of them hosting the FPGA, power management and host communication units dealing with one sensor half, is shown on the left, while the right part is the daughterboard on which the LinoSPAD2 integrated circuit itself is mounted.

Download Full Size | PDF

 figure: Fig. 4.

Fig. 4. Representative architectures for the reconfigurable imaging sensor concept as demonstrated with the LinoSPAD2 system: (a) basic photon counting with per-pixel counters, (b) time-gated photon counting, implemented with several counters and a time gate that is shifted with 18-ps resolution, (c) time-correlated single-photon counting with a TDC shared among four pixels and raw TDC data read-out, and (d) TDC array as in (c) with additional on-the-fly histogram generation for reduced bandwidth and increased photon efficiency.

Download Full Size | PDF

2.3 TDC architecture with smart multiplexing

Time-correlated single-photon counting (TCSPC) applications rely on TDCs built with the FPGA’s carry chains [3539]. Other recent TDC techniques on FPGA involve the use of gray code oscillators with the aim of decreasing the amount of FPGA resources utilized, with the drawback of worsening the time resolution. The authors in [40] overcame the low resolution problem with a new matrix scheme achieving 20.97 ps TDC bin in a 16 nm Ultrascale+ FPGA device. The carry chain-based TDC or tapped-delay-line (TDL) has become common in fields sharing the need for high timing precision [36,41,42]. It relies on delay lines constructed from dedicated carry logic available in programmable logic elements, providing the fastest paths from one register input to the next, and thus the best timing resolution possible, at the same time minimizing non-linearities. The LinoSPAD2 delay line is sampled at the highest possible frequency in Spartan 6 FPGAs, 400 MHz, thereby minimizing its length and reducing timing degradation. We employ 35 carry elements with four outputs each, resulting in a total of 140 bit samples. This is sufficient to cover 2.5 ns of the clock period for all operating conditions, i.e. taking into account timing differences due to different locations in the FPGA, as well as variations due to operating temperature or voltage. A schematic of the TDC architecture is shown in Fig. 7 for clarity.

TDC arrays are available within the LinoSPAD2 system in several variants. Given that TDC blocks are relatively large, a single TDC per pixel is not feasible in a Spartan 6 FPGA. This leds to three main time-resolved implementations, with the basic one coupling 64 TDCs to 64 SPADs in a fixed topology. This is already sufficient for a number of use cases. In a second implementation (Fig. 4(c)), a single TDC is shared amongst four pixels and the raw timestamps are made directly available to the user. In this case the 64 TDCs can be dedicated to groups of 64 SPADs at any time, moving to the next group of 64 pixels in a round-robin fashion (sequential access), or serving all pixels simultaneously by means of smart 4:1 multiplexers (shared access). The latter is obviously more efficient but harder to implement, given that conflicts need to be avoided. The underlying principle and approach are shown in Fig. 5. The novel smart multiplexers are implemented in the LinoSPAD2 FPGA architecture as shown in Fig. 6. They have been placed-and-routed in close proximity with the FPGA TDLs with ad-hoc constraints. The pulses coming from 4 SPAD pixels are processed in parallel by a modified OR tree to retain the address of the SPAD which fired, and by an LUT programmed with the SPAD combinations for which the timestamp is valid. All the SPAD pulse processing is implemented in the same configurable logic block (CLB), together with the synchronizing flip flops, which are sampled by the fast 400 MHz clock. The output of these provides the SPAD codes together with the TDC valid bit which determines if the TDC code, sampled at the same clock edge, is recorded or discarded. Up to now we implemented only the condition for which we validate the timestamp if just a single SPAD fired during the clock period and we leave for future implementations the possibility of reprogramming this block to implement smart processing of the SPAD logic pulses, such as enabling different levels of coincidence. The overall final maximum TDC rate is 133 MHz, with a maximum range of 4.5 ms (28 bits) and an average LSB of 20 ps. In a typical configuration, the FPGA utilization levels of the logic, register and memory resources are of 71%, 43% and 94%, respectively.

 figure: Fig. 5.

Fig. 5. (Left) Pulse shrinking circuit schematic for a single pixel, tailored for implementation in an FPGA. Input pulses are shortened to 2.5-5.0 ns. (Right) Pulse shrinking circuit timing diagram. The shrunk pulses of four pixels are combined using an OR-gate to share the same delay line. This leads to higher event rates and lower pulse losses due to temporal overlap of the input signals.

Download Full Size | PDF

 figure: Fig. 6.

Fig. 6. Smart multiplexing scheme implemented in a single FPGA CLB to allow concurrent operation of four different input SPADs while detecting possible timing conflicts. The delayline is triggered when one of the four inputs has a rising edge. The address of the input pulse is determined by the Address high/low signals and the timestamp is validated if TDC valid is asserted.

Download Full Size | PDF

 figure: Fig. 7.

Fig. 7. Time-to-digital converter FPGA architecture illustrating one of the smart multiplexing-enabled TDCs. Only seven Carry4 configurable logic blocks (CLBs) are shown on the left side for simplicity, while in the real FPGA architecture the delayline is composed of 35 elements. The delayline input comes from the input CLB, shown at the bottom of the image, which hosts the smart multiplexing scheme detailed in Fig. 5 and Fig. 6. The carry chain outputs are converted to binary by means of a fast thermometer encoder in a pipelined fashion. The final timestamps are then resynchronized to a 133 MHz clock domain by means of a rate reducing block.

Download Full Size | PDF

When individual timestamps are not needed, system efficiency can be further enhanced by adding an on-FPGA on-the-fly histogram generation engine as in Fig. 4(d), with a corresponding substantial read-out bandwidth reduction. Typical configurations are 1024 bins of 16 bits each or equivalent, limited by the available memory blocks.

3. System characterization

The sensor has been characterized over 256 pixels in terms of breakdown voltage, system SPTR, PDP, crosstalk and afterpulsing, and over 512 pixels in terms of dark count rate (DCR). The DCR measurements were performed by means of counters implemented on FPGA, while for the SPTR measurements the TDC array was employed.

3.1 Breakdown voltage

The SPAD breakdown voltage levels have been measured in the dark, counting the first DCR digital pulses produced by the pixels. To characterize the breakdown voltage in temperature, the IC was inserted in a temperature chamber with temperatures ranging from -65$^{\circ }$ to +60$^{\circ }$. For each temperature the SPAD voltage has been swept with a 0.05 V granularity from the expected breakdown voltage -0.5 V to +0.5 V and 10 measurements have been acquired for each voltage point. A linear fit has then been employed to determine the intercept with the voltage axis in the counts over SPAD voltage graph. The results of this characterization are shown in Fig. 8. A correction of 0.6 V has been applied, to take into account the inverter threshold voltage in the pixel.

 figure: Fig. 8.

Fig. 8. Mean SPAD breakdown voltage characterization over temperature; the error bars correspond to the standard deviation. Inset: breakdown voltage distribution over the array at 20$^{\circ }$C. For other temperatures the distribution was found to be similar around the mean breakdown voltage.

Download Full Size | PDF

3.2 Dark count rate

After having determined the breakdown voltages across the array the DCR was measured. The system was enclosed in a black box during the measurements to remove all spurious light sources and to avoid that the sensor warms up during the integration time. An exposure time of 1 s was set and 10 acquisitions were performed per excess bias voltage, and then averaged. The resulting cumulative DCR distribution over the full array is shown in Fig. 9 at room temperature and for different excess bias voltages, for a representative sensor. In Fig. 9 the median DCR of the same pixel array is plotted from 1 to 7 V of excess bias voltage at 20$^o$C. Results for these SPADs are in agreement with previously reported SPADs in this technology [28,30].

 figure: Fig. 9.

Fig. 9. (Left) Cumulative DCR distribution over the full array for different excess bias voltages, measured at 20$^o$C. A number of pixels, corresponding to the missing parts of the plots on the left, were inactive due to electrical contact issues. (Right) Median DCR over the full array as a function of the excess bias voltage, measured at room temperature.

Download Full Size | PDF

3.3 Sensitivity

The PDP has been characterized between 320 nm and 960 nm at different excess bias voltages with the optical set-up detailed in [31,43]. A wide spectrum Xenon lamp was employed as a light source and fed to a monochromator and then to an integrating sphere. A reference photodiode (Hamamatsu S2281) was used for the intensity calibration. After calibration, the LinoSPAD2 chip was placed in the setup, at distance of about 14 cm from the output port, and characterized. Sufficiently low light levels over the sensor were employed to avoid pile-up effects, which could potentially alter the SPAD photo-sensitivity curve. Before scanning the wavelengths, a DCR measurement was performed at a specific excess bias and then subtracted. In Fig. 10 the PDP as a function of wavelength is shown for different excess bias voltages. To improve the sensor light detection efficiency, microlenses have been designed and imprinted on the sensor [44]. Fig. 11 shows a comparison between the PDE with and without microlenses and a plot of the concentration factor (CF) with respect to wavelength. This parameter is measured as the ratio of the count rates with microlenses and without and it determines the ability of the lenses to focus light onto the SPAD photosensitive area.

 figure: Fig. 10.

Fig. 10. Detector PDP as a function of the excess bias voltage and wavelength (mean values over the pixel array). The peak PDP is at 520 nm with a value of 53 % @ $V_{ex}$ = 5 V.

Download Full Size | PDF

 figure: Fig. 11.

Fig. 11. (Left) Mean photon detection efficiency at $V_{EX}$ = 5 V and $V_{diode}$ = 1.4 V with and without microlenses (left) and corresponding concentration factors (right). Error bars correspond to the standard deviation of the measurements. The transmission reduction in the NUV, below 400 nm, is caused by the microlens material. Reprinted with permission from [45] © The Optical Society. (Right) Photo-response non uniformity (PRNU) over the SPAD array at 520 nm and $V_{EX}$ = 5 V. Left: PDE, right: corresponding CF. The linear fit coefficients are indicated in the plot, together with the $R^2$ parameter to indicate fit quality. The upper red plot was measured for a sensor with microlenses, while the lower one for a sensor without. Reprinted with permission from [45] © The Optical Society.

Download Full Size | PDF

Fig. 11 shows the average CF over all pixels, computed for every wavelength. Applying different excess bias voltages does not influence the mean CF value of 2.3, which is slightly below the simulated value of 2.9 for NA $\approx$ 0.177 [45]. This is likely due to manufacturing imperfections in the microlens shape and/or microlens-to-pixel misalignment errors. The pixel-to-pixel variation shown in Fig. 11, measured at a fixed wavelength close to the PDP peak, features a small gradient and a relative PDE change of 4.0% peak-to-peak between the two sides of the array, according to a linear fit. This applies to both arrays shown, i.e. one with microlenses (top) and one without (bottom). Finally in Fig. 12 the full uniformity data over the array at 5 V excess bias is reported at a fixed excess bias voltage.

 figure: Fig. 12.

Fig. 12. PDP as a function of wavelength and pixel number, at $V_{EX}$ = 5 V and $V_{diode}$ = 1.4 V.

Download Full Size | PDF

3.4 Afterpulsing

Afterpulsing has been characterized for different pixels in the array, soldering short wires directly onto the daughterboard PCB and analyzing the pixel traces by means of a fast oscilloscope (LeCroy WaveMaster 813 Zi-B) using active probes. The histogram of the inter-arrival times for a reference pixel is shown in Fig. 13. A dead time of 18 ns was observed in the trace, which is in agreement with the value of the $\mathrm {RC}$ constant of the pixel passive quenching/recharge circuit of 12.6 ns according to our design, see Fig. 2. However no double exponential behavior has been observed, and therefore no afterpulsing value is reported. This is due to the passive recharge mechanism employed for this pixel. As observed in [46], afterpulsing in this type of SPAD is not zero, however it cannot be observed with an experimental setup characterized by relatively large parasitic capacitance in parallel to the SPAD.

 figure: Fig. 13.

Fig. 13. Inter-arrival time plot for a reference single pixel in the array.

Download Full Size | PDF

3.5 TDC characterization

The TDC non-linearity characterization was performed by exposing the sensor to uncorrelated light distributed uniformly over the 400 MHz sampling clock period in the so-called code density test. Fig. 14 shows the results of the test; a point of measure indicates the recorded bin size in ps for each of the 1024 bins in the TDC. A mean LSB of 18 ps with a standard deviation of 13 ps was measured in any given delay line with respect to a reference. An on-FPGA calibration step is used to compensate for the variability of all the bin sizes, thereby reducing differential and integral non-linearity. The non-linearity correction scheme consists in a histogram count redistribution (bottom part of Fig. 14) by means of a matrix multiplication. The uncalibrated histogram is multiplied by a matrix of weights computed from the code density test results. The counts are redistributed in such a way that the equivalent size of the new calibrated TDC bins after correction is:

$$\mathrm{B}_{cal} = \frac{\mathrm{T}_{TDC}}{\mathrm{N}_{bin,cal}}$$
where $\mathrm {B}_{cal}$ is the width of the new TDC bins (22.9 ps in our case), $\mathrm {T}_{TDC}$ is the TDC period (20 ns) and $N_{bin,cal}$ is the new total number of bins (800). In our case $\mathrm {N}_{bin,cal}$, i.e. after calibration, is smaller than its counterpart before calibration due to the elimination of empty bins while maintaining the same TDC range. The timing skew is then compensated in the firmware by using a correlated light source, whereby the FPGA is programmed with a peak-finding algorithm to determine the fixed time difference between each channel. This allows to time-align all channels, as will also be apparent in the IRF curves shown in Section G. The overall skew magnitude range is of $\pm$8 ns.

 figure: Fig. 14.

Fig. 14. Time-to-digital converter bin sizes over a reference TDC delay line on the FPGA, obtained by means of a code density test. The top figure shows the raw uncalibrated TDC bins, while the bottom graph shows the bin sizes after non-linearity correction implemented directly on the FPGA.

Download Full Size | PDF

3.6 Crosstalk

The overall system crosstalk was characterized using the TDCs on the FPGA, following the methodology detailed in [47]. After the non-linearity calibration and an offline skew correction, the sensor was illuminated with light from a halogen lamp, attenuated by means of ND filters and a diffuser, in order to achieve a uniform illumination over the SPAD array, with an excess bias voltage of 6 V. We then analyzed the raw timestamps for pixel pairs, one "aggressor" and one "victim", by counting the number of events detected by the victim inside a 500 ps time window after the aggressor event. We then subtracted from this value the mean number of crosstalk events at larger timescales (multiple victim events are compared and histogrammed in a time window $\approx 100$ ns, taken after $5$ ns wrt the aggressor event), which are uniformly distributed. The result was divided by the total number of events recorded by the two selected pixels over the 48.4 seconds acquisition time. Fig. 15 reports the crosstalk probability map for 16 adjacent pixels (corresponding to 16 TDC channels in the FPGA). The highest pixel crosstalk level which we observed was $\approx 0.19 {\% }$, which is consistent with other measurements in the same technology and for similar SPAD sizes.

 figure: Fig. 15.

Fig. 15. Crosstalk probability matrix at 6 V excess bias over a section of the LinoSPAD2 array. Inset: example of crosstalk probability for five pixels adjacent to pixel 3.

Download Full Size | PDF

3.7 System Instrument response function

The system SPTR has been characterized employing a femtosecond pulsed laser (NKT photonics ORIGAMI O-08LP) with a typical pulse duration $\leq$160 fs and a center-wavelength of 780 nm. The laser has been synchronized with the LinoSPAD2 board using a high bandwidth RF cable and the laser light power has been attenuated using neutral density filters before illuminating the SPAD array using a diffuser. Neutral density filters ensure that the array is operating in photon-starved mode, so as to avoid pile-up. The TDCs on the FPGA were used for these measurements, and their raw timestamps calibrated as previously described. A Gaussian fit around the peak (+-$\sigma _e$ bins) was applied to the data to extract the FWHM values of all the pixels. A value of $\sigma _e = 10$ was chosen to take into account only the laser peak for the calculation. In Fig. 16 three histograms of the jitter FWHM values are shown over the array for three different excess bias voltages. As expected, the timing performance improves with increased excess bias, in accordance with standard SPAD theory and practice. In addition to the FWHM distribution over the array, the normalized, calibrated IRFs of selected pixels (251 out of 256) are also shown in Fig. 16.

 figure: Fig. 16.

Fig. 16. Normalized IRF curves for multiple SPAD pixels as acquired by the TDCs on FPGA, after correction for non-linearities and skews between the pixels. Inset: FWHM distribution over the array.

Download Full Size | PDF

3.8 System power consumption

The overall system power consumption has been characterized for different modes of operation, corresponding to representative light levels. The results are shown in Table 1 for some key voltage supplies (see also Fig. 2 and Fig. 3). The main power consumption source is the 5V plug powering the FPGA, the voltage regulators, the flash memory and the USB communication interface. When the light power onto the SPAD active area increases, the higher activity impacts the I/O blocks of the chip as well as the FPGA inputs, which draw more current.

Tables Icon

Table 1. System power consumption for different illumination scenarios without optics on the sensor.

4. LinoSPAD2 time-resolved applications

The versatility of the LinoSPAD2 camera system is illustrated in several time-resolved applications, whether operated in a synchronous or asynchronous manner. Table 2 lists all those applications for which the LinoSPAD concept was used, to date, along with core firmware requirements. The first five are specific to LinoSPAD2 and rely in one way or another on its hardware and/or firmware key features (high sensitivity combined with parallel, virtually noiseless read-out), to improve the acquisition speed and/or the signal quality. The applications are diverse in complexity, from compressive Raman, where high-speed counting over all pixels and low DCR are needed, to heralded spectroscopy, where a selected pixel range can be addressed and a fixed TDC topology is acceptable. In other applications, such as quantum random number generators, pixels must be truly independent and as many as possible are needed to increase throughput. Conversely, in quantum astrometry/spectroscopy, there is a need to detect correlations between distant pixels, and shared TDC access is necessary. In this case the smart multiplexing scheme is a key feature enabling the direct measurement of time and energy (i.e., wavelength or frequency) over the full sensor, in a spectrometer set-up, for simultaneous single photons, thereby combining excellent temporal and spectral resolution. Indeed, without smart multiplexing only a portion of the array would be usable for this application. Initial results have already been reported at only a factor of ten above the Heisenberg Uncertainty Principle limit of $\hbar$/2 for energy and time [48]. In addition, having a $\approx$30% peak PDE, which is also one of the highest reported for linear SPAD arrays, decreases by almost five times the integration time (compared to the original LinoSPAD) given that the probability of coincidence events is proportional to the square of the PDE.

Tables Icon

Table 2. LinoSPAD2 (*) or LinoSPAD (**) reconfigurable cameras reference applications, together with specific firmware requirements. Reference columns: further information on the application and (when available) specific results obtained so far. N/A: not yet available or work in progress. NLOS: non-line-of-sight imaging, QRNG: Quantum Random Number Generation.

The last argument applies as well to other applications where detection in coincidence plays a prominent role, such as room temperature single-particle heralded spectroscopy of quantum dots and lead halide perovskite nanocrystals [49,50]. In this case, LinoSPAD2 made it possible to extend the temporal resolution limit of standard spectrometers by several orders of magnitude. Furthermore, the combination of low DCR and high PDE enabled compressive Raman sensing [51,52] at a very short effective pixel dwell time of 800 ns, which would be hardly feasible in presence of high DCR as the weak spontaneous Raman signal would be buried by noise. This achievement enabled an effective parallelization of single-pixel compressive Raman and represented an important step towards high-speed chemical imaging. Finally, in all cases low crosstalk is appreciated to reduce spurious events and coincidences, in heralded spectroscopy in particular, and in quantum imaging in general.

Up to now we did focus on fixed firmware functionality, designed for the target application; however, FPGAs can be reconfigured dynamically at run-time if need be. For instance, an increase of processing power for selected tasks may be needed and several firmware IP blocks could be added for further enhancements, typically implementing real-time post-processing steps. Examples include time-of-arrival estimation, peak intensity extraction, lifetime estimation in FLIM, or autocorrelator arrays for diffusion estimation [53] or $g_2$ calculations [54]. Additional performance achievements are expected by moving to more advanced FPGA platforms, although the existing platform has proven to be sufficient for most applications. Two main boards are in principle needed to address the full sensor. However, user feedback has shown that in a number of cases a restricted amount of pixels is perfectly sufficient. If so, they can selected amongst those featuring the least high-DCR pixels.

5. Conclusions

LinoSPAD2 is a time-resolved camera featuring 512 SPADs arranged in a linear configuration and connected directly, in a fully parallel manner, to an FPGA. This solution enables a reconfigurable and highly customizable camera for a variety of very diverse applications. The pixel pitch is 26.2 $\mu$m, while the SPADs have a median DCR of 80 cps, a peak PDP of 53%, an intrinsic SPTR of 50 ps FWHM, and an average IRF at system level of 106.3 ps at 5 V excess bias. The LinoSPAD concept has been totally redesigned: the sensor itself is twice as long as the original one, the PDP has improved by more than 60%, also thanks to the novel high-voltage compact monostable front-end, the PDE has been multiplied by $\approx$ 2.3 times by means of custom imprinted microlenses, and the median DCR reduced by at least 50 times. This has been coupled to enhanced firmware and software, in particular the smart multiplexing option, to allow the exploration of new applications. Serving all pixels simultaneously was for example simply not possible before. Key results are reported in Table 3 and compared with literature.

Tables Icon

Table 3. State-of-the-art comparison of large line format CMOS SPAD sensors with either on-chip or FPGA-based data acquisition and processing.

LinoSPAD2 is a hardware-reconfigurable imager featuring high-sensitivity and natively digital single-photon detection in standard CMOS. This allows to process photons a few nanoseconds after detection with possibly complex processing done practically in situ. This firmware- and software-defined camera system is a new paradigm in image sensors, enabled by the speed and digital nature of SPADs. It will support future single-photon-aware and quantum imaging and new emerging applications.

Funding

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (20QT21-187716; 200021-166289).

Acknowledgments

The authors would like to acknowledge Francesco Gramuglia for the fruitful technical discussions about the PDP setup and the femtosecond laser setup, Mathieu Moulin (EPFL) for system testing, Gur Lubin, Ron Tenne and Dan Oron (Weizmann Institute of Science), Sergei Kulkov and Jakub Jirsa (Czech Technical University, Prague), as well as Andrei Nomerotski (Brookhaven National Laboratory), for characterization assistance.

Disclosures

CB: PI Imaging Technology SA (I,S), EC: Fastree3D SA (I,S) and PI Imaging Technology SA (I,S).

Data availability

Data may be obtained from the authors upon reasonable request.

References

1. E. Charbon, “Single-photon imaging in complementary metal oxide semiconductor processes,” Phil. Trans. R. Soc. A. 372(2012), 20130100 (2014). [CrossRef]  

2. Y. Maruyama, J. Blacksberg, and E. Charbon, “A 1024×8, 700-ps Time-Gated SPAD Line Sensor for Planetary Surface Exploration With Laser Raman Spectroscopy and LIBS,” IEEE J. Solid-State Circuits 49(1), 179–189 (2014). [CrossRef]  

3. A. Rochas, M. Gani, B. Furrer, et al., “Single photon detector fabricated in a complementary metal–oxide–semiconductor high-voltage technology,” Rev. Sci. Instrum. 74(7), 3263–3270 (2003). [CrossRef]  

4. C. Bruschini, H. Homulle, I. M. Antolovic, et al., “Single-photon avalanche diode imagers in biophotonics: review and outlook,” Light: Sci. Appl. 8(1), 87 (2019). [CrossRef]  

5. C. Niclass, C. Favi, T. Kluter, et al., “A 128×128 Single-Photon Image Sensor With Column-Level 10-Bit Time-to-Digital Converter Array,” IEEE J. Solid-State Circuits 43(12), 2977–2989 (2008). [CrossRef]  

6. C. Veerappan, J. Richardson, R. Walker, et al., “A 160× 128 single-photon image sensor with on-pixel 55ps 10b time-to-digital converter,” in 2011 IEEE International Solid-State Circuits Conference, (IEEE, 2011), pp. 312–314.

7. I. Nissinen, J. Nissinen, P. Keränen, et al., “A 2 × (4) × 128 Multitime-Gated SPAD Line Detector for Pulsed Raman Spectroscopy,” IEEE Sens. J. 15(3), 1358–1365 (2015). [CrossRef]  

8. L. H. Braga, L. Pancheri, L. Gasparini, et al., “A CMOS mini-SiPM detector with in-pixel data compression for PET applications,” in 2011 IEEE Nuclear Science Symposium Conference Record, (IEEE, 2011), pp. 548–552.

9. R. K. Henderson, N. Johnston, S. W. Hutchings, et al., “5.7 A 256×256 40nm/90nm CMOS 3D-stacked 120dB dynamic-range reconfigurable time-resolved SPAD imager,” in 2019 IEEE International Solid-State Circuits Conference-(ISSCC), (IEEE, 2019), pp. 106–108.

10. S. Mandai and E. Charbon, “A 4×4×416 digital SiPM array with 192 TDCs for multiple high-resolution timestamp acquisition,” J. Instrum. 8(05), P05024 (2013). [CrossRef]  

11. A. Carimatto, S. Mandai, E. Venialgo, et al., “11.4 A 67,392-SPAD PVTB-compensated multi-channel digital SiPM with 432 column-parallel 48ps 17b TDCs for endoscopic time-of-flight PET,” in 2015 IEEE International Solid-State Circuits Conference-(ISSCC) Digest of Technical Papers, (IEEE, 2015), pp. 1–3.

12. A. R. Ximenes, P. Padmanabhan, M.-J. Lee, et al., “A modular, direct time-of-flight depth sensor in 45/65-nm 3-D-stacked CMOS technology,” IEEE J. Solid-State Circuits 54(11), 3203–3214 (2019). [CrossRef]  

13. I. Cusini, D. Berretta, E. Conca, et al., “Historical Perspectives, State of Art and Research Trends of SPAD Arrays and Their Applications (Part II: SPAD Arrays),” Frontiers in Physics p. 606, (2022).

14. S. Ma, S. Gupta, A. C. Ulku, et al., “Quanta burst photography,” ACM Trans. Graph. 39(4), 79 (2020). [CrossRef]  

15. A. T. Erdogan, R. Walker, N. Finlayson, et al., “A 16.5 giga events/s 1024 × 8 SPAD line sensor with per-pixel zoomable 50ps-6.4ns/bin histogramming TDC,” in 2017 Symposium on VLSI Circuits, (2017), pp. C292–C293.

16. C. Zhang, S. Lindner, I. M. Antolovic, et al., “A 30-frames/s, 252 × 144 SPAD Flash LiDAR With 1728 Dual-Clock 48.8-ps TDCs, and Pixel-Wise Integrated Histogramming,” IEEE J. Solid-State Circuits 54(4), 1137–1151 (2019). [CrossRef]  

17. C. Niclass, C. Favi, T. Kluter, et al., “Single-Photon Synchronous Detection,” IEEE J. Solid-State Circuits 44(7), 1977–1989 (2009). [CrossRef]  

18. J. Richardson, R. Walker, L. Grant, et al., “A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in 2009 IEEE Custom Integrated Circuits Conference, (IEEE, 2009), pp. 77–80.

19. E. Charbon, C. Bruschini, C. Veerappan, et al., “SPADnet: A fully digital, networked approach to MRI compatible PET systems based on deep-submicron CMOS technology,” in 2013 IEEE Nuclear Science Symposium and Medical Imaging Conference (2013 NSS/MIC), (IEEE, 2013), pp. 1–5.

20. D. Portaluppi, E. Conca, and F. Villa, “32×32 CMOS SPAD imager for gated imaging, photon timing, and photon coincidence,” IEEE J. Sel. Top. Quantum Electron. 24(2), 1–6 (2018). [CrossRef]  

21. S. Burri, Y. Maruyama, X. Michalet, et al., “Architecture and applications of a high resolution gated SPAD image sensor,” Opt. Express 22(14), 17573–17589 (2014). [CrossRef]  

22. H. Homulle, F. Powolny, P. Stegehuis, et al., “Compact solid-state CMOS single-photon detector array for in vivo NIR fluorescence lifetime oncology measurements,” Biomed. Opt. Express 7(5), 1797–1814 (2016). [CrossRef]  

23. L. Gasparini, M. M. García, M. Zarghami, et al., “A reconfigurable 224 × 272-pixel single-photon image sensor for photon timestamping, counting and binary imaging at 30.0 − μm pitch in 110nm CIS technology,” in ESSCIRC 2022-IEEE 48th European Solid State Circuits Conference (ESSCIRC), (IEEE, 2022), pp. 189–192.

24. S. W. Hutchings, N. Johnston, I. Gyongy, et al., “A reconfigurable 3-D-stacked SPAD imager with in-pixel histogramming for flash LIDAR or high-speed time-of-flight imaging,” IEEE J. Solid-State Circuits 54(11), 2947–2956 (2019). [CrossRef]  

25. S. Burri, H. Homulle, C. Bruschini, et al., “LinoSPAD: a time-resolved 256×1 CMOS SPAD line sensor system featuring 64 FPGA-based TDC channels running at up to 8.5 giga-events per second,” in Optical Sensing and Detection IV, vol. 9899 (SPIE, 2016), pp. 57–66.

26. S. Burri, C. Bruschini, and E. Charbon, “Applications of a reconfigurable SPAD line imager (Conference Presentation),” in Photonic Instrumentation Engineering V, vol. 10539 (SPIE, 2018), p. 1053904.

27. C. Bruschini, S. Burri, E. Bernasconi, et al., “LinoSPAD2: a 512×1 linear SPAD camera with system-level 135-ps SPTR and a reconfigurable computational engine for time-resolved single-photon imaging,” in Quantum Sensing and Nano Electronics and Photonics XIX, vol. 12430 (SPIE, 2023), pp. 126–135.

28. C. Veerappan and E. Charbon, “A low dark count p-i-n diode based SPAD in CMOS technology,” IEEE Trans. Electron Devices 63(1), 65–71 (2016). [CrossRef]  

29. S. Burri, C. Bruschini, and E. Charbon, “LinoSPAD: a compact linear SPAD camera system with 64 FPGA-based TDC modules for versatile 50 ps resolution time-resolved imaging,” Instruments 1(1), 6 (2017). [CrossRef]  

30. I. M. Antolovic, C. Bruschini, and E. Charbon, “Dynamic range extension for photon counting arrays,” Opt. Express 26(17), 22234–22248 (2018). [CrossRef]  

31. F. Gramuglia, M.-L. Wu, C. Bruschini, et al., “A low-noise CMOS SPAD pixel with 12.1 ps SPTR and 3 ns dead time,” IEEE J. Sel. Top. Quantum Electron. 28, 1–9 (2022). [CrossRef]  

32. E. Conca, S. Riccardo, V. Sesta, et al., “Design of a 16 × 16 fast-gated SPAD imager with 16 integrated shared picosecond TDCs for non-line-of-sight imaging,” in Emerging Imaging and Sensing Technologies for Security and Defence IV, vol. 11163 (SPIE, 2019), pp. 25–32.

33. E. A. Webster, J. A. Richardson, L. A. Grant, et al., “A single-photon avalanche diode in 90-nm CMOS imaging technology with 44% photon detection efficiency at 690 nm,” IEEE Electron Device Lett. 33(5), 694–696 (2012). [CrossRef]  

34. E. A. G. Webster and R. K. Henderson, “Single photon avalanche diode for CMOS circuits,” (2015). US Patent 9,178,100.

35. C. Favi and E. Charbon, “A 17ps Time-to-Digital Converter Implemented in 65nm FPGA Technology,” in Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, (Association for Computing Machinery, New York, NY, USA, 2009), FPGA ’09, p. 113–120.

36. H. Menninga, C. Favi, M. W. Fishburn, et al., “A multi-channel, 10ps resolution, FPGA-based TDC with 300MS/s throughput for open-source PET applications,” in 2011 IEEE Nuclear Science Symposium Conference Record, (IEEE, 2011), pp. 1515–1522.

37. F. Dadouche, T. Turko, W. Uhring, et al., “New design-methodology of high-performance TDC on a low cost FPGA targets,” Sensors Transducers 193, 123 (2015).

38. H. Chen and D. D.-U. Li, “Multichannel, low nonlinearity time-to-digital converters based on 20 and 28 nm FPGAs,” IEEE Trans. Ind. Electron. 66(4), 3265–3274 (2019). [CrossRef]  

39. W. Xie, Y. Wang, H. Chen, et al., “128-channel high-linearity resolution-adjustable time-to-digital converters for LiDAR applications: Software predictions and hardware implementations,” IEEE Trans. Ind. Electron. 69(4), 4264–4274 (2022). [CrossRef]  

40. Y. Wang, W. Xie, H. Chen, et al., “Low-Hardware Consumption, Resolution-Configurable Gray Code Oscillator Time-to-Digital Converters Implemented in 16 nm, 20 nm, and 28 nm FPGAs,” IEEE Trans. Ind. Electron. 70(4), 4256–4266 (2023). [CrossRef]  

41. J. Song, Q. An, and S. Liu, “A high-resolution time-to-digital converter implemented in field-programmable-gate-arrays,” IEEE Trans. Nucl. Sci. 53(1), 236–241 (2006). [CrossRef]  

42. W. Pan, G. Gong, and J. Li, “A 20-ps time-to-digital converter (TDC) implemented in field-programmable gate array (FPGA) with automatic temperature correction,” IEEE Trans. Nucl. Sci. 61(3), 1468–1473 (2014). [CrossRef]  

43. G. Zappalà, F. Acerbi, A. Ferri, et al., “Set-up and methods for SiPM Photo-Detection Efficiency measurements,” J. Instrum. 11(08), P08014 (2016). [CrossRef]  

44. J. M. Pavia, M. Wolf, and E. Charbon, “Measurement and modeling of microlenses fabricated on single-photon avalanche diode arrays for fill factor recovery,” Opt. Express 22(4), 4202–4213 (2014). [CrossRef]  

45. C. Bruschini, I. M. Antolovic, F. Zanella, et al., “Challenges and Prospects for Multi-chip Microlens Imprints on front-side illuminated SPAD Imagers,” Opt. Express 31(13), 21935–21953 (2023). [CrossRef]  

46. C. Veerappan, “Single-photon avalanche diodes for cancer diagnosis,” Ph.D. thesis, TU Delft (2016).

47. G. Lubin, R. Tenne, I. M. Antolovic, et al., “Quantum correlation measurement with single photon avalanche diode arrays,” Opt. Express 27(23), 32863–32882 (2019). [CrossRef]  

48. J. Jirsa, S. Kulkov, R. A. Abrahao, et al., “Fast spectrometer near the Heisenberg limit with direct measurement of time and frequency for multiple single photons,” arXiv, arXiv:2304.11999 (2023). [CrossRef]  

49. G. Lubin, G. Yaniv, M. Kazes, et al., “Resolving the controversy in biexciton binding energy of cesium lead halide perovskite nanocrystals through heralded single-particle spectroscopy,” ACS Nano 15(12), 19581–19587 (2021). [CrossRef]  

50. G. Lubin, R. Tenne, A. C. Ulku, et al., “Heralded spectroscopy reveals exciton–exciton correlations in single colloidal quantum dots,” Nano Lett. 21(16), 6756–6763 (2021). [CrossRef]  

51. C. Gentner, S. Burri, E. Charbon, et al., “Parallelized compressive spontaneous Raman imaging via SPAD arrays (Conference Presentation),” in High-Speed Biomedical Imaging and Spectroscopy VIII, (SPIE, 2023), p. PC123900K.

52. C. Gentner, S. Burri, E. Charbon, et al., “Compressive Raman microspectroscopy parallelized by single-photon avalanche diode arrays,” arXiv, arXiv:2301.07709 (2023). [CrossRef]  

53. J. Buchholz, J. W. Krieger, G. Mocsár, et al., “FPGA implementation of a 32×32 autocorrelator array for analysis of fast image series,” Opt. Express 20(16), 17767–17782 (2012). [CrossRef]  

54. M. A. Wayne, E. J. Sie, A. C. Ulku, et al., “Massively parallel, real-time multispeckle diffuse correlation spectroscopy using a 500×500 SPAD camera,” Biomed. Opt. Express 14(2), 703–713 (2023). [CrossRef]  

55. G. Lubin, R. Tenne, A. C. Ulku, et al., “Heralded spectroscopy: a new single-particle probe for nanocrystal photophysics,” in Single Molecule Spectroscopy and Superresolution Imaging XVI, vol. 12386 (SPIE, 2023), p. 1238603.

56. A. Nomerotski, P. Stankus, A. Slosar, et al., “Quantum-assisted optical interferometers: instrument requirements,” in Optical and Infrared Interferometry and Imaging VII, vol. 11446 (SPIE, 2020), pp. 290–306.

57. P. Stankus, A. Nomerotski, A. Slosar, et al., “Two-photon amplitude interferometry for precision astrometry,” arXiv, arXiv:2010.09100 (2020). [CrossRef]  

58. F. Soldevila, J. Dong, E. Tajahuerce, et al., “Fast compressive Raman bio-imaging via matrix completion,” Optica 6(3), 341–346 (2019). [CrossRef]  

59. B. Sturm, F. Soldevila, E. Tajahuerce, et al., “High-sensitivity high-speed compressive spectrometer for Raman imaging,” ACS Photonics 6(6), 1409–1415 (2019). [CrossRef]  

60. M. J. Fernée, P. Tamarat, and B. Lounis, “Cryogenic single-nanocrystal spectroscopy: reading the spectral fingerprint of individual CdSe quantum dots,” J. Phys. Chem. Lett. 4(4), 609–618 (2013). [CrossRef]  

61. Y. Ma, L. Huang, C. Sen, et al., “Light-field tomographic fluorescence lifetime imaging microscopy,” Research Square, rs.3.rs-2883279 (2023). [CrossRef]  

62. X. Feng and L. Gao, “Ultrafast light field tomography for snapshot transient and non-line-of-sight imaging,” Nat. Commun. 12(1), 2179 (2021). [CrossRef]  

63. X. Feng, Y. Ma, and L. Gao, “Compact light field photography towards versatile three-dimensional vision,” Nat. Commun. 13(1), 3333 (2023). [CrossRef]  

64. A. Stanco, D. G. Marangon, G. Vallone, et al., “Efficient random number generation techniques for CMOS single-photon avalanche diode array exploiting fast time tagging units,” Phys. Rev. Res. 2(2), 023287 (2020). [CrossRef]  

65. H. Fürst, H. Weier, S. Nauerth, et al., “High speed optical quantum random number generation,” Opt. Express 18(12), 13029–13037 (2010). [CrossRef]  

66. M. Herrero-Collantes and J. C. Garcia-Escartin, “Quantum random number generators,” Rev. Mod. Phys. 89(1), 015004 (2017). [CrossRef]  

67. M. O’Toole, F. Heide, D. B. Lindell, et al., “Reconstructing transient images from single-photon sensors,” in Proceedings of the IEEE conference on computer vision and pattern recognition, (2017), pp. 1539–1547.

68. M. Buttafava, J. Zeman, A. Tosi, et al., “Non-line-of-sight imaging using a time-gated single photon avalanche diode,” Opt. Express 23(16), 20997–21011 (2015). [CrossRef]  

69. J. Léonard, N. Dumas, J.-P. Caussé, et al., “High-throughput time-correlated single photon counting,” Lab Chip 14(22), 4338–4343 (2014). [CrossRef]  

70. A. I. Skilitsi, T. Turko, D. Cianfarani, et al., “Towards sensitive, high-throughput, biomolecular assays based on fluorescence lifetime,” Methods Appl. Fluoresc. 5(3), 034002 (2017). [CrossRef]  

71. A. T. Erdogan, R. Walker, N. Finlayson, et al., “A CMOS SPAD line sensor with per-pixel histogramming TDC for time-resolved multispectral imaging,” IEEE J. Solid-State Circuits 54(6), 1705–1719 (2019). [CrossRef]  

72. T. Talala, E. Parkkinen, and I. Nissinen, “CMOS SPAD Line Sensor With Fine-Tunable Parallel Connected Time-to-Digital Converters for Raman Spectroscopy,” IEEE J. Solid-State Circuits 58(5), 1350–1361 (2022). [CrossRef]  

73. J. M. Pavia, M. Scandini, S. Lindner, et al., “A 1× 400 backside-illuminated SPAD sensor with 49.7 ps resolution, 30 pJ/sample TDCs fabricated in 3D CMOS technology for near-infrared optical tomography,” IEEE J. Solid-State Circuits 50(10), 2406–2418 (2015). [CrossRef]  

74. P. Keranen, K. Maatta, and J. Kostamovaara, “Wide-range time-to-digital converter with 1-ps single-shot precision,” IEEE Trans. Instrum. Meas. 60(9), 3162–3172 (2011). [CrossRef]  

Data availability

Data may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (16)

Fig. 1.
Fig. 1. LinoSPAD2 IC micrograph detail. Microlenses are imprinted on top of the SPADs, with the alignment crosses drawn with the final metal layer on the array side (one cross is visible on the left).
Fig. 2.
Fig. 2. LinoSPAD2 pixel circuit with voltage waveforms highlighted, enabling individual SPAD readout and 1:1 coupling to external FPGAs. This configuration enables high excess bias voltages thanks to AC coupling to logic circuits and higher photon count rates. $V_{a}$ indicates the peak voltage of the inverter input, for which the maximum is set by $V_{diode}$ .
Fig. 3.
Fig. 3. LinoSPAD2 system. One of the two custom-made motherboards, each of them hosting the FPGA, power management and host communication units dealing with one sensor half, is shown on the left, while the right part is the daughterboard on which the LinoSPAD2 integrated circuit itself is mounted.
Fig. 4.
Fig. 4. Representative architectures for the reconfigurable imaging sensor concept as demonstrated with the LinoSPAD2 system: (a) basic photon counting with per-pixel counters, (b) time-gated photon counting, implemented with several counters and a time gate that is shifted with 18-ps resolution, (c) time-correlated single-photon counting with a TDC shared among four pixels and raw TDC data read-out, and (d) TDC array as in (c) with additional on-the-fly histogram generation for reduced bandwidth and increased photon efficiency.
Fig. 5.
Fig. 5. (Left) Pulse shrinking circuit schematic for a single pixel, tailored for implementation in an FPGA. Input pulses are shortened to 2.5-5.0 ns. (Right) Pulse shrinking circuit timing diagram. The shrunk pulses of four pixels are combined using an OR-gate to share the same delay line. This leads to higher event rates and lower pulse losses due to temporal overlap of the input signals.
Fig. 6.
Fig. 6. Smart multiplexing scheme implemented in a single FPGA CLB to allow concurrent operation of four different input SPADs while detecting possible timing conflicts. The delayline is triggered when one of the four inputs has a rising edge. The address of the input pulse is determined by the Address high/low signals and the timestamp is validated if TDC valid is asserted.
Fig. 7.
Fig. 7. Time-to-digital converter FPGA architecture illustrating one of the smart multiplexing-enabled TDCs. Only seven Carry4 configurable logic blocks (CLBs) are shown on the left side for simplicity, while in the real FPGA architecture the delayline is composed of 35 elements. The delayline input comes from the input CLB, shown at the bottom of the image, which hosts the smart multiplexing scheme detailed in Fig. 5 and Fig. 6. The carry chain outputs are converted to binary by means of a fast thermometer encoder in a pipelined fashion. The final timestamps are then resynchronized to a 133 MHz clock domain by means of a rate reducing block.
Fig. 8.
Fig. 8. Mean SPAD breakdown voltage characterization over temperature; the error bars correspond to the standard deviation. Inset: breakdown voltage distribution over the array at 20 $^{\circ }$ C. For other temperatures the distribution was found to be similar around the mean breakdown voltage.
Fig. 9.
Fig. 9. (Left) Cumulative DCR distribution over the full array for different excess bias voltages, measured at 20 $^o$ C. A number of pixels, corresponding to the missing parts of the plots on the left, were inactive due to electrical contact issues. (Right) Median DCR over the full array as a function of the excess bias voltage, measured at room temperature.
Fig. 10.
Fig. 10. Detector PDP as a function of the excess bias voltage and wavelength (mean values over the pixel array). The peak PDP is at 520 nm with a value of 53 % @ $V_{ex}$ = 5 V.
Fig. 11.
Fig. 11. (Left) Mean photon detection efficiency at $V_{EX}$ = 5 V and $V_{diode}$ = 1.4 V with and without microlenses (left) and corresponding concentration factors (right). Error bars correspond to the standard deviation of the measurements. The transmission reduction in the NUV, below 400 nm, is caused by the microlens material. Reprinted with permission from [45] © The Optical Society. (Right) Photo-response non uniformity (PRNU) over the SPAD array at 520 nm and $V_{EX}$ = 5 V. Left: PDE, right: corresponding CF. The linear fit coefficients are indicated in the plot, together with the $R^2$ parameter to indicate fit quality. The upper red plot was measured for a sensor with microlenses, while the lower one for a sensor without. Reprinted with permission from [45] © The Optical Society.
Fig. 12.
Fig. 12. PDP as a function of wavelength and pixel number, at $V_{EX}$ = 5 V and $V_{diode}$ = 1.4 V.
Fig. 13.
Fig. 13. Inter-arrival time plot for a reference single pixel in the array.
Fig. 14.
Fig. 14. Time-to-digital converter bin sizes over a reference TDC delay line on the FPGA, obtained by means of a code density test. The top figure shows the raw uncalibrated TDC bins, while the bottom graph shows the bin sizes after non-linearity correction implemented directly on the FPGA.
Fig. 15.
Fig. 15. Crosstalk probability matrix at 6 V excess bias over a section of the LinoSPAD2 array. Inset: example of crosstalk probability for five pixels adjacent to pixel 3.
Fig. 16.
Fig. 16. Normalized IRF curves for multiple SPAD pixels as acquired by the TDCs on FPGA, after correction for non-linearities and skews between the pixels. Inset: FWHM distribution over the array.

Tables (3)

Tables Icon

Table 1. System power consumption for different illumination scenarios without optics on the sensor.

Tables Icon

Table 2. LinoSPAD2 (*) or LinoSPAD (**) reconfigurable cameras reference applications, together with specific firmware requirements. Reference columns: further information on the application and (when available) specific results obtained so far. N/A: not yet available or work in progress. NLOS: non-line-of-sight imaging, QRNG: Quantum Random Number Generation.

Tables Icon

Table 3. State-of-the-art comparison of large line format CMOS SPAD sensors with either on-chip or FPGA-based data acquisition and processing.

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

B c a l = T T D C N b i n , c a l
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.