Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Streak tube imaging lidar with kilohertz laser pulses and few-photons detection capability

Open Access Open Access

Abstract

Lidar using active light illumination is capable of capturing depth and reflectivity information of target scenes. Among various technologies, streak tube imaging lidar (STIL) has garnered significant attention due to its high resolution and excellent precision. The echo signals of a STIL system using single laser pulse are often overwhelmed by noise in complex environments, making it difficult to discern the range of the target. By combining high-frequency laser pulses with the repetitive sweep circuit, the STIL system enables efficient detection of few-photons signal in weak-light environments. Additionally, we have developed a robust algorithm for estimating the depth and reflectivity images of targets. The results demonstrate that this lidar system achieves a depth resolution better than 0.5 mm and a ranging accuracy of 95 um. Furthermore, the imaging of natural scenes also validates the exceptional 3D imaging capability of this system.

© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Acquiring three-dimensional coordinates of target scenes holds significant importance in various fields, such as environmental monitoring, smart city development, spatial perception, and national defense security [13]. Currently, while multiple scanning modalities exist to obtain target position information, the STIL system has garnered considerable attention in three-dimensional imaging due to its high detection sensitivity and excellent resolution [4,5].

The STIL system enables full waveform imaging by digitizing the echo laser signal, achieving high-precision 3D imaging through laser emission and corresponding echo signal detection [6]. It utilizes the streak tube as the core detector based on the principle of direct time-of-flight (dTOF). The streak camera captures ultrafast signals in a spatially ordered manner, thus acquiring the temporal information of the target echo signal from laser. With excellent temporal resolution, it achieves high distance resolution and serves as a solid foundation for 3D imaging. In contrast, normal high-speed cameras only capture photon intensity information and cannot accurately determine the arrival time of photons, thus being unable to directly obtain target distance information. Since F.K. Knight [7] first reported the 3D image acquisition of targets using STIL system in 1989, significant progress has been made in theoretical analysis and experimental validation [810]. In engineering applications, Fugro's rapid airborne multibeam mapping system (RAMMS) is used for simultaneously acquiring high-resolution bathymetry and topographic data, as well as the required situational awareness imagery and associated position [11].

STIL system relies on the high-speed sweep circuit to convert time information into spatial information and typically uses single laser pulse imaging. In long-range and high-attenuation environments, the low signal-to-noise ratio (SNR) of STIL system poses difficulties in detecting the target signal position. To improve imaging quality in complex environments such as fog or water, the STIL systems have to emit a higher energy laser pulse, which poses significant challenges[ 12]. Accordingly, researchers have proposed several improvement strategies, such as using high-energy laser sources with carrier modulation [13], optimizing detector performance [4], and developing enhanced signal processing algorithms [14]. These methods have shown potential in enhancing the detection capability of STIL systems in complex environments. However, when the number of echo photons from single laser pulse reaches the level of single-photon, the signal is often drowned out by noise, which poses limitations to these methods.

Therefore, we designed the streak tube imaging lidar system with multiple laser pulses (MLP-STIL) to improve the SNR, enabling few-photons echo signals detection. Specifically, we accumulate the echo signals from multiple laser pulses into one frame image using the high repetitive sweep circuit. This method operates in the high-frequency mode with photon probability detection to enhance the signal identification capability. The MLP-STIL system has the following advantages:

  • (i) By accumulating the response from multiple laser pulses, it is possible to identify target signals in the few-photons scenario.
  • (ii) The streak tube incorporates a microchannel plate (MCP) that can amplify single-photon [1517]. This allows for a large dynamic range in the number of photons received in single laser pulse, without wasting multiple photons as in photon counting mode [18,19].
  • (iii) The streak tube simultaneously possesses high temporal resolution (approximately 5ps) and spatial resolution, enabling the acquisition of high-quality 3D images.

To the best of our knowledge, this paper first designed the MLP-STIL system with high resolution and precision. The system aims to address the detection challenges of weak echo signals in complex environments. The proposed algorithm demonstrates the ability to obtain high-quality images, showcasing exceptional robustness. Through a series of successful experiments, the system's outstanding 3D imaging capabilities have been convincingly validated.

2. Methodology

2.1 Theoretical analysis

In the MLP-STIL system, the core components of the signal detection module include an imaging lens, streak tube, MCP, and complementary metal-oxide-semiconductor (CMOS) camera. The imaging lens and streak tube are responsible for the detection and imaging tasks, while the MCP and CMOS camera primarily amplify and capture the signals. For MLP-STIL system, each frame of the streak image is composed of multiple laser pulse echo signals superimposed together. To accurately describe the system's characteristics, we define the key parameter: the effective acquisition count η, which represents the number of laser pulses recorded within one frame exposure time of the CMOS camera.

$$\eta = T.{f_{RF}}$$
where T is the acquisition time of one frame image and fRF is the laser frequency. During one frame exposure time, the detector not only responds to the ballistic light signal Ie from the target echoes but also receives various noises. These noise sources include the background noise Nb generated by the scattered light and the detector's intrinsic noise Nd. Consequently, the total energy expression can be represented as follows:
$${I_{totle}} = \sum\limits_{i = 1}^\eta {{I_{{e_i}}}} + \sum\limits_{j = 1}^\eta {{N_{{b_j}}}} + {N_d}$$

The SNR is used to measure the ability of the system to detect and identify targets under specific conditions. In the case of the STIL system, the SNR is defined as the ratio of the ballistic light power density to the noise power density. The expression for SNR is:

$$SN{R_\eta } = \frac{{E{{\left( {\sum\limits_{i = 1}^\eta {{I_{{e_i}}}} } \right)}^2}}}{{E{{\left( {\sum\limits_{j = 1}^\eta {{N_{{b_j}}}} + {N_d}} \right)}^2}}}$$

In the STIL system, the detector's noise exhibits diversity. During the photon capture process, there is uncertainty due to random fluctuations in the collection of echo photons by the photocathode of the streak tube, leading to the photon shot noise, which follows the Poisson process [20,21]. Additionally, in the signal amplification process, the MCP exhibit non-uniformity in their light response to incident light intensity, known as the photo response Non-Uniformity (PRNU), which can be modeled using the Gaussian distribution [22].

$${N_{d1}} = \sum\limits_{i = 1}^\eta {\alpha Poisson({I_{{e_i}}})} + \sum\limits_{j = 1}^\eta {g{I_{{e_j}}}Normal(0,{\delta _{PRNU}})} $$

During the process of recording signals in CMOS, the imperfect photoelectric conversion due to the defects in the light sensor can lead to the generation of photon shot noise, dark noise, and readout noise [23].

$${N_{d2}} = \sum\limits_{i = 1}^\eta {\beta Poisson({I_{{e_i}}})} + T.{D_{dark}} + Normal(0,{\delta _{read}})$$

Here, α and β are proportionality factors, and g represents the gain coefficient. The SNR with an effective acquisition count of η in one frame image can be expressed as:

$$SN{R_\eta } = \frac{{{{\overline {{I_e}} }^2}}}{{{\alpha ^2}\overline {{I_e}} + {g^2}{{\overline {{I_e}} }^2}{\delta _{PRNU}}^2 + {\beta ^2}\overline {{I_e}} + \frac{{{D_{dark}}^2}}{{{f_{RF}}^2}} + \frac{{{\delta _{read}}^2}}{{{\eta ^2}}} + {{\overline {{N_b}} }^2}}}$$

When the single-laser echo signal Ie is high, the photon shot noise and PRNU noise, which are multiplicative noises associated with the signal, significantly impact the SNR of the system. However, when the single-laser echo signal Ie is very low, approaching zero, the expression for SNR is:

$$\mathop {SN{R_\eta }}\limits_{\overline {{I_e}} \to 0} = \frac{{{{\overline {{I_e}} }^2}}}{{\frac{{{D_{dark}}^2}}{{{f_{RF}}^2}} + \frac{{{\delta _{read}}^2}}{{{\eta ^2}}} + {{\overline {{N_b}} }^2}}}$$

By using the narrowband optical filter, the background noise can be effectively filtered out, reducing $\bar{\textrm{N}}_{\textrm{b}}\to 0$. Additionally, the higher fRF can weaken the impact of dark noise Ddark on the SNR. As a result, when weak signal echoes are present, the readout noise of CMOS becomes the primary factor affecting the SNR. Therefore, increasing the count η of laser detections can significantly improve the SNR of the system, which is of great importance for accurately identifying weak signals.

2.2 Systematic imaging principle

The STIL system, known for its excellence in linear scanning imaging, relies on the precise localization of target distance using the dTOF of the laser pulse. In the presence of complex imaging environments, the single laser echo signal is often overwhelmed by background noise and detector noise, making it challenging to accurately identify the signal position. Figure 1(a) showcases the architecture of the MLP-STIL system, while Fig. 1(e) presents the physical configuration of the system. Table 1 summarizes the main parameters of the system. Specifically, we employed a high-performance laser as the active illumination source. The laser model used in this study is PH1-20 (PHAROS). Despite the laser system being equipped with an external clock signal, there is a significant jitter of approximately 470 ps between the output electrical signal and the optical laser signal. This jitter can greatly impact the accuracy and reliability of range measurements in the laser ranging system that utilizes the dTOF principle. To mitigate the influence of laser jitter on the system's range error, we utilized optical triggering to generate electrical signals. The implementation involved using a beam splitter to divide 10% of the pulse energy to excite the photodiode. Through precise control of the delay generator, we ensured high-precision time synchronization between the sweep circuit and the laser echo signal. The remaining 90% of the laser beam was directed through a beam expander (model: BE05) to reduce the divergence angle and enhance spatial resolution. The diaphragm was used to constrain the size of the laser spot after beam expander. The polarizing plate (P1) allowed only horizontally polarized light to pass through. The polarized light was transformed into a fan-shaped beam through a cylindrical lens. The laser emission path and echo reception path of the system were coaxially transmitted via a polarizing beam splitter (PBS). Additionally, the quarter-wave plate was utilized to convert linearly polarized light into circularly polarized light. To acquire the 3D coordinate information of the target, we employed a uniaxial mirror to quickly step the laser beam angle and rotate it onto the target. The echo signal was examined by a polarizing plate (P2) and imaged onto the detector through an objective lens and a bandpass filter.

 figure: Fig. 1.

Fig. 1. (a) Schematic diagram of the high-frequency laser and few-photos echo streak tube imaging lidar system. The system includes: pulsed laser; beam splitter(BS); beam expander (BE05); diaphragm(D); polarizer(P1,P2); cylinder lens(CL); polarization beam splitter (PBS); objective lens(OL); bandpass filter(BPF); reflector(R);quarter wave plate(QWP); galvanometer mirror(GM), avalanche Photo Diode(APD); CMOS; digital delay generator; sweep circuit; streak tube; signal generator. (b) The working principle schematic of the streak tube. (c) Target echo image recorded by CMOS sensor. (d) The time sequence diagram of CMOS and galvanometer mirror. (e) Physical drawing of the system.

Download Full Size | PDF

Tables Icon

Table 1. Summary of the main system parameters.

Figure 1(b) illustrates the working process of the faint echo signal in the streak tube. The photocathode converts the optical signal into the electrical signal, and then the photoelectrons pass through the accelerating grid and focusing electrode to enter the deflector plate. The high-voltage pulse sweep circuit is responsible for deflecting the photoelectrons to different positions on the fluorescent screen, thereby converting time information into spatial positions. Additionally, the image intensifier module enhances the weak signal for better detection of low light. Finally, the CMOS records the signal positions on the fluorescent screen and reflects the signal intensity through grayscale values. Figure 1(c) displays the real target echo image recorded by CMOS. The horizontal axis represents the spatial positions of the illuminated target, while the vertical axis represents the corresponding time information. Different distances correspond to different time information, enabling the estimation of the target's distance by identifying the signal position. To avoid motion distortion, a signal generator generates electrical signals of specific frequencies to synchronize the CMOS sensor and the galvanometer mirror. As a result, the imaging speed of the system is determined by the frequency set on the signal generator. Figure 1(d) provides the detailed depiction of the timing sequence of the CMOS and galvanometer mirror. Here, t1 represents the acquisition time of the CMOS, during which multiple laser echo signals are captured. t2 is the static time of the galvanometer mirror, set to be greater than t1 to ensure that the CMOS sensor accurately captures the echo signals from the same position within one frame. t3 represents the rotation time of the galvanometer mirror, and its rotation speed can be adjusted through software control. Since the mirror rotates over a small angle each time, the rotation speed can be very fast, typically much smaller than t2. Finally, the sum of t3 and t2 determines the imaging frame rate of the system, which remains consistent with the frequency set on the signal generator.

2.3 Reconstruction algorithm

Accurately identifying the temporal position of target signals is indeed stressful, particularly in the presence of weak signal echoes. Traditional STIL systems primarily rely on peak localization algorithm [24] and maximum likelihood estimation (MLE) algorithm [25] for signal identification. The peak localization algorithm involves analyzing the peak position of the signal distribution to determine the target's distance information. The maximum likelihood estimation algorithm treats all recorded gray values as target signals, ignoring the influence of noise, and determines the target distance by calculating the centroid position of the signal distribution, also known as the centroid weighting method. However, the imaging process of the streak tube and CMOS sensor inevitably introduces various types of noise, and these noise sources significantly interfere with weak signals, leading to increased depth errors when attempting to identify weak signals. As a result, accurate target distance information cannot be provided.

To address this issue, we introduce the cross-correlation algorithm. Noise exhibits random distribution in the time series, while signals demonstrate apparent clustering characteristics. In theory, without the influence of noise, the response functions of strong and weak echoes are the same. Strong echoes typically have higher signal strengths, resulting in more pronounced and easily identifiable responses in the STIL system. Conversely, weak echoes may have lower signal strengths, making them more susceptible to noise and resulting in degraded response waveforms that are more challenging to accurately detect and measure. This algorithm significantly enhances ranging accuracy when dealing with weak signals, enabling accurate identification of target signals even in noisy environments. Specifically, the cross-correlation algorithm determines the temporal position by convolving the time-intensity curve with the system imaging response function (IRF) and identifying the position with the highest correlation. IRF is the system imaging response function obtained through calibration using the high-energy laser. When the high-energy laser irradiates the target, the target reflects laser signals and the CMOS captures strong echo signals. We further select one pixel and observe its grayscale variation in the time domain. After normalization, this grayscale variation can be used as the IRF. It characterizes the response of the system and is related to factors such as pulse duration, pulse shape, and modulation. The reflectivity image is represented by the corresponding gray value. The mathematical expression for this is shown:

$$t(x,y) = \mathop {\arg \max }\limits_t \sum\limits_{i = 1}^{{N_{tb}}} {{g_{t + i}}(x,y)} \times IR{F_i}$$

In this context, g(x,y) represents the grayscale variation of the pixel in the time dimension. x and y represent the horizontal and vertical pixel positions in the reconstructed image, respectively. Ntb denotes the number of time bins. The $\otimes$ symbol represents the convolution operation. Figure 2 illustrates the process of the cross-correlation algorithm in identifying the positions of weak echo signals. In Fig. 2(a), the original grayscale variation of the pixel in the time dimension is presented. Due to noise interference, the signal is masked, making it difficult to directly identify the peak time position. Figure 2(b) illustrates the system's IRF obtained through pre-calibration with high-energy laser (full-width half-maximum of 11.26ps). Figure 2(c) shows the result of convolving the original time-grayscale curve with the IRF, revealing clear peak points. At the peak point, the horizontal axis t(x,y) denotes the temporal position of the target pixel, and the vertical axis r(x,y) reflects the reflectivity of the target pixel.

 figure: Fig. 2.

Fig. 2. (a) Real detected signals with its gray value variation in the time dimension. (b) The priori system imaging response function obtained by high-energy laser. (c) Time-gray value curve obtained after the cross-correlation algorithm.

Download Full Size | PDF

Furthermore, due to the non-uniformity of the light response at different positions of the streak tube photocathode and the differences in reflectivity at different positions of the target, the cross-correlation method may fail under extremely low-SNR conditions. Therefore, after applying the cross-correlation algorithm to determine the target contour, further processing is necessary to ensure reliability. Leveraging the natural spatial correlation of objects, we select a neighboring set of (2x + 1)*(2y + 1) pixels for adaptive median filtering to eliminate the influence of outliers. By calculating the absolute rank difference between the spatial point and neighboring elements, combined with a predefined threshold, we can determine whether the spatial point belongs to noise. For pixels identified as noise, we employ histogram statistics to analyze the values within the small matrix and obtain the most probable true value to replace the noisy value. Finally, the obtained depth and reflectivity images are subjected to smoothing using the neural network-based FFTNET algorithm [26], resulting in a clearer and more accurate image.

3. Experiments and results

To validate the performance of the MLP-STIL system, we conducted target imaging experiments of approximately 4m distance. To obtain clear echo signals, we used the laser power of 0.8mW and an acquisition time of 700ms to reconstruct the target image, which is ground truth as the reference image. We attenuated the laser power to 23uW using a neutral density filter as the weak-light environment and observed the reconstruction results at the acquisition times of 50ms, 300ms, and 1000ms (corresponding to η are 500, 3000, and 10000) to test the system's few-photons imaging capability. This laser parameter was set for all experiments. The strength of the target echo signal was quantified using the signal-to-background ratio (SBR), which is expressed as:

$$SBR = \frac{1}{{mn}}\sum\limits_{i = 1}^m {\sum\limits_{j = 1}^n {\frac{{\sum\limits_{t = 1}^{{N_{tb}}} {{g_{i,j}}(t) - {N_{tb}}\ast \overline {Noise} } }}{{{N_{tb}}\ast \overline {Noise} }}} }$$
where m and n represent the horizontal and vertical pixel numbers of the reconstruction image, respectively. gi,j(t) is the gray value in the time dimension. $\overline {\textrm{Noise}} $ is the average background noise of the system. Figure 3 illustrates the gray value in the time dimension of different acquisition times in the weak-light environment. When laser pulses η = 500, the SBR drops to a low value of 0.091, and the signal is submerged in noise. As shown in Fig. 2, it is only through the implementation of the cross-correlation algorithm that the position of the echo signals can be determined. Therefore, in the case of η = 1, the reality is that the echo signal from the laser is already extremely weak, making it impossible to identify the signal using any method. It is evident that as the effective laser detection count η increases, the position of the echo signal becomes more prominent, and the SBR correspondingly improves.

 figure: Fig. 3.

Fig. 3. The gray value of the system in the time dimension for different η (500, 3000 and 10000).

Download Full Size | PDF

3.1 Depth resolution test

During the flight of photoelectrons from the photocathode to the fluorescent screen, there is a temporal distortion (TD) effect caused by the variation in transit time from different photocathode positions [4]. This effect directly affects the depth measurement, causing planar targets to appear curved in depth. To eliminate the impact on depth measurement, pre-calibration and algorithmic correction are crucial (all depth images shown below have been corrected). Figure 4(a) demonstrates the streak image obtained by the STIL system for the planar target, clearly showing the temporal displacement between the edges and the center. Furthermore, Fig. 4(b) quantifies the curvature of the planar target through the cross-correlation algorithm, revealing a difference of approximately 2.6 millimeters between the edges and the center. With the corrected depth, there is a small error between the edge and the center.

 figure: Fig. 4.

Fig. 4. (a)The temporal distortion image of a planar target by the STIL system. (b)The bending effect obtained through the cross-correlation algorithm and the curve after correction of the planar target

Download Full Size | PDF

Depth resolution was evaluated with a 140mm × 160 mm 3D printed resin target board. The board was arranged with a series different depth of 10mm × 10mm square platforms, as detailed in Fig. 5(a) (depth unit in mm). The STIL system is a linear detection ranging system that relies on the rotation of the galvanometer mirror to obtain complete 3D information about the target. After capturing one frame streak image, the rotation angle of the galvanometer mirror is precisely set through control software. In our experiments, this angle was set to 0.11 mrad. It is worth noting that the lateral field of view of the system mainly depends on the focal length of the objective lens, while the vertical field of view is determined by the number of frames in the streak images. Under the condition of the 4 m detection distance, the lateral field of view covered by 1042 pixels is 76.6 mm. Meanwhile, by capturing 178 frames of streak images, the vertical field of view is extended to 78.3 cm, which is sufficient to cover the target area of our 3D printed board in the experiment. When high-energy laser illumination, the elevation map of the depth board was reconstructed, as shown in Fig. 5(b). Different colors represent different depths, and it is evident that each small square platform is accurately differentiated with a gradient change in color. Figure 5(c) displays the reconstructed 178*1042 pixels depth image, which is considered the true reference image. To validate the depth resolution, we selected an area of interest with a 0.5mm depth gradient. From Fig. 5(c), we extracted a row of pixels along the horizontal and vertical directions and plotted the corresponding curves in Fig. 5(d) and Fig. 5(e). By observing these curves, we can intuitively assess the depth resolution of the system in the horizontal and vertical directions. The results indicate that the depth resolution of the system exceeds 0.5mm.

 figure: Fig. 5.

Fig. 5. (a) 3D printed resin depth board. (b) Elevation map obtained by cross-correlation algorithm under high-energy laser. (c) Depth image as the reference. (d) Horizontal curve of 0.5mm depth gradient. (e) Vertical curve of 0.5mm depth gradient.

Download Full Size | PDF

To validate the system's detection capability in weak-light environments and the potential of the algorithm to recognize weak signals, we conducted depth-resolution experiments using low-energy lasers. We quantified the error between the estimated depth $t$ and the reference depth t using the root mean square error (RMSE), expressed as follows:

$$RMSE(t,\mathop t\limits^{\widehat{}} ) = \sqrt {\frac{1}{{mn}}\sum\limits_{i = 1}^m {\sum\limits_{j = 1}^n {{{({t_{i,j}} - {{\mathop t\limits^{\widehat{}} }_{i,j}})}^2}} } }$$

In weak-light conditions, the imaging results using different algorithms with varying acquisition times are shown in Fig. 6. From the figure, it can be observed that as the acquisition time increases, the SBR gradually enhances, resulting in reduced errors in the reconstructed depth images. Additionally, our proposed reconstruction method exhibits significant improvements in depth accuracy compared to the peak algorithm and MLE algorithm. In the case of 50 ms acquisition time, the peak and MLE methods are unable to distinguish the depth differences of the target, while our method can accurately reconstruct the distinct depths of the target. Furthermore, it is noteworthy that our reconstructed image with 50ms acquisition time has smaller distance errors compared to the traditional methods with 1000ms acquisition time.

 figure: Fig. 6.

Fig. 6. Depth resolution results using different algorithms for acquisition times of 50ms, 300ms and 1000ms respectively.

Download Full Size | PDF

For the STIL system, theoretically, the depth resolution is equivalent to one time bin width, which is 331 fs. According to the dTOF principle: d = c.Δt/2, we can calculate the theoretical limit of depth resolution to be 50 um. However, during the actual detection and imaging process, even for the same plane, there may be slight differences in the peak time positions. Therefore, range accuracy is a crucial metric for lidar systems. To improve range accuracy, the system employs the optical triggering method, effectively minimizing the influence of laser jitter on range measurements. Moreover, the high-frequency probability working mode further reduces adverse effects caused by jitter in the high-voltage sweep circuit. To quantitatively evaluate the system's range accuracy, we conducted 300 repeated experiments of depth measurements at the same location. We used the cross-correlation algorithm to determine the time positions of the same pixel points. Due to the presence of noise, there are certain differences in the peak time positions processing each frame streak image. We performed statistical analysis using the sample standard deviation (SD) and range of error (ROE) [18,27]. The depth SD can be considered as the actual depth resolution for the lidar ranging system. The ROE is defined as the difference between the maximum and minimum measured depths. The range accuracy with acquisition times of 50ms, 300ms, and 1000ms is shown in Fig. 7. When the acquisition time is 50 ms, the ROE is 1.688 mm, and the SD of the 300 measurements is 0.262 mm. This indicates that under low-SBR conditions, there is a significant error and relatively high dispersion in the single-point ranging results. As the laser pulse number increases, the SBR is improved significantly. This leads to a more accurate determination of the peak time positions and a gradual reduction in the ROE. At the same time, the SD also decreases, indicating that the ranging results for the same point become more precise and consistent. When the acquisition time is 1000 ms, the ROE is reduced to 0.646 mm, and the SD decreases to 0.095 mm. This result fully demonstrates the high range accuracy of the MLP-STIL system.

 figure: Fig. 7.

Fig. 7. The range accuracy for the cross-correlation algorithm with acquisition times of 50ms, 300ms, and 1000ms. The first row shows the 300 experiments results of depth measurements. The vertical axis represents the discrete distribution of ROE. The second row shows the histogram statistics for the 300 results, including the central tendency and SD of the ranging data.

Download Full Size | PDF

3.2 Reflectivity resolution test

We utilized the USAF1951 negative standardized resolution chart to evaluate the system's reflectivity resolution, as depicted in Fig. 8(a). The normalized reflectivity image of 118 × 690 pixels was acquired using the high-energy laser, as shown in Fig. 8(b). To thoroughly evaluate the system's reflectivity resolution performance in weak-light conditions, we reconstructed the reflectivity images at different acquisition times, as shown in Fig. 8(c). We introduced two key metrics to assess the quality of the reconstructed images: peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). The results demonstrate that as the SBR increases, both PSNR and SSIM improve, indicating a gradual convergence of the reconstructed images towards the ground truth. Compared to traditional algorithms, our method exhibits significant improvement in target detail reconstruction and feature extraction. To determine the system's spatial resolution with 400 mm focal length of the objective lens, Fig. 9 displays the horizontal intensity profile and vertical intensity profile extracted from the reference image in Fig. 8(b). Analysis reveals that at 400mm focal length of the objective lens and the distance of 4m, the system achieves a horizontal resolution of 0.125mrad and a vertical resolution of 0.099mrad.

 figure: Fig. 8.

Fig. 8. (a) The physical image of USAF1951 negative standardized resolution chart. (b) Real reference reflectivity image. (c) Reflectivity images of the resolution plate using different algorithms with different acquisition times.

Download Full Size | PDF

 figure: Fig. 9.

Fig. 9. The horizontal and vertical intensity profiles extracted from the reference real image.

Download Full Size | PDF

3.3 Natural scenes

To comprehensively evaluate the performance of the MLP-STIL system in natural scene imaging, we conducted experiments on two natural scenes: the geometric composition and the David model, as shown in Fig. 10. We reconstructed the natural scene using different algorithms, and Fig. 11 and Fig. 12 respectively display the depth images and normalized reflectivity images with different acquisition times. The geometric data consists of 230 x1440 pixels, while the David data consists of 360 x1436 pixels.

 figure: Fig. 10.

Fig. 10. Two natural scenes. (a) Geometric composition. (b) David model.

Download Full Size | PDF

 figure: Fig. 11.

Fig. 11. The depth images of geometric composition and David model. (a) Acquisition time: 50ms. (b) Acquisition time: 300ms. (c) Acquisition time: 1000ms.

Download Full Size | PDF

 figure: Fig. 12.

Fig. 12. The normalized reflectivity images of geometric composition and David model. (a) Acquisition time: 50ms. (b) Acquisition time: 300ms. (c) Acquisition time: 1000ms.

Download Full Size | PDF

We observed that the peak algorithm and maximum likelihood estimation (MLE) algorithm performed poorly in depth estimation and reflectance reconstruction due to weak echo signals being overwhelmed by noise. However, with our method, even at an acquisition time of 50ms, we were able to observe the shapes of the geometric objects and the contours of the David model. Table 2 presents the results of RMSE for depth estimation, PSNR, and SSIM for reflectivity estimation for natural scene data at different acquisition times. The table shows variations in the SBR for the geometric objects and the David model at the same acquisition time due to material differences. Figure 13 illustrates the variations of RMSE for depth, PSNR, and SSIM for reflectivity. The data indicates that our method exhibits significant advantages over the peak algorithm and MLE algorithm at different acquisition times. However, we must acknowledge that under extremely low SBR conditions, such as the SBR of 0.021 for the geometric composition, our method has limitations, resulting in larger errors in the reconstructed images. This is primarily due to the severe shortage of photons in a significant portion of the scene, leading to errors in determining the positions of signal echoes. In contrast, when the SBR for the David model is 0.051, indicating a slight increase in the number of echo photons, the reconstruction of depth and reflectivity images performs better. Although our method still faces challenges in handling extremely weak echo signals, it demonstrates certain superiority compared to traditional methods, enabling the recovery of the basic shapes.

 figure: Fig. 13.

Fig. 13. The trend of RMSE in mm for depth and PSNR in dB and SSIM for reflectivity by different algorithms on the geometric combination and David model.

Download Full Size | PDF

Tables Icon

Table 2. The results of RMSE in mm for depth and PSNR in dB and SSIM for reflectivity by different algorithms on Geometric combination and David model.

4. Conclusion

In this article, we design the 10 KHz streak tube imaging lidar system with high resolution and accuracy. By combining high-repetition streak tube imaging and our algorithm, we successfully enhance the 3D imaging capability of the STIL system under low-light conditions. Specifically, the streak images are formed by accumulating the echo signals from multiple laser pulses, effectively improving the SBR and enhancing the imaging quality. Moreover, our proposed algorithm demonstrates superior performance in identifying the temporal location of the target echo signal compared to traditional peak and MLE algorithms at different acquisition times.

Through a series of experiments, we validate the outstanding performance of the lidar system. Firstly, depth imaging experiments using a 3D printed board show that the system achieves a range resolution exceeding 0.5mm. Secondly, results from 300 distance tests at the same position demonstrate the depth standard deviation of only 95um. Additionally, reflectivity imaging of the USAF1951 negative resolution target further confirms that the system has a horizontal resolution of 0.125 mrad and a vertical resolution of 0.099 mrad at the lens focal length of 400 mm. Finally, through imaging experiments of complex geometric objects and the David model, we fully validate the high-precision 3D imaging capability in low-light environments of the MLP-STIL system.

This article focuses on obtaining target depth and reflectivity images by accumulating multiple laser pulses and algorithmic recognition, enabling recognition in few-photons scenarios. This mechanism has the potential to facilitate advancements in remote sensing and underwater exploration. In the future, we will delve into the research of higher-frequency sweep circuits and image restoration with lower SBR in extreme environments, aiming to further enhance the system's performance and expand its application range.

Funding

National Natural Science Foundation of China (62075236); Youth Innovation Promotion Association of the Chinese Academy of Sciences (2020397); Shaanxi Provincial Key R&D Program (2024GX-YBXM-090).

Disclosures

The authors declare that there are no conflicts of interest related to this article

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. C. Mallet and F. Bretar, “Full-waveform topographic lidar: State-of-the-art,” ISPRS Journal of photogrammetry and remote sensing. 64(1), 1–16 (2009). [CrossRef]  

2. J Gao, J Sun, J Wei, et al., “Research of underwater target detection using a slit streak tube imaging lidar,” 2011 Academic International Symposium on Optoelectronics and Microelectronics Technology. IEEE, pp. 240–243 (2011).

3. G. Zhou and M. Xie, “Coastal 3-D morphological change analysis using LiDAR series data: a case study of Assateague Island National Seashore,” J. Coastal Res. 25(2), 435–447 (2009). [CrossRef]  

4. D. Hui, D. Luo, L. Tian, et al., “A compact large-format streak tube for imaging lidar,” Rev. Sci. Instrum. 89(4), 4 (2018). [CrossRef]  

5. F. Zong, J. Zhang, B. Guo, et al., “Research of mini steak tube with planar image plane for laser three-dimensional scanning Lidar,” Optik 185, 1157–1162 (2019). [CrossRef]  

6. G. Mandlburger, H. Lehner, and N. Pfeifer, “A comparison of single photon and full waveform lidar,” ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci. 4, 397–404 (2019). [CrossRef]  

7. F. K. Knight, D. I. Klick, D. P. Ryan-Howard, et al., “Three-dimensional imaging using a single laser pulse,” Laser radar IV. SPIE, Vol. 1103. (1989).

8. B. C. Redman, A. J. Griffis, and E. B. Schibley, “Streak tube imaging lidar (STIL) for 3-D imaging of terrestrial targets,” Proc. of 2000 Meeting of the MSS Specialty Group on Active EO Systems, 11–13, (2000).

9. Z. Chen, R. F. Rongwei Fan, G. Y. Guangchao Ye, et al., “Depth resolution improvement of streak tube imaging lidar system using three laser beams,” Chin. Opt. Lett. 16(4), 041101 (2018). [CrossRef]  

10. T. Luo, R. Fan, Z. Chen, et al., “Deblurring streak image of streak tube imaging lidar using Wiener deconvolution filter,” Opt. Express 27(26), 37541–37551 (2019). [CrossRef]  

11. D. Ventura, “Coastal zone mapping with the world’s first airborne multibeam bathymetric lidar mapping system,” Hydrogr. Nachrichten. 115, 48–53 (2020).

12. J. Gao, J. Sun, and Q. Wang, “Experiments of ocean surface waves and underwater target detection imaging using a slit Streak Tube Imaging Lidar,” Optik 125(18), 5199–5201 (2014). [CrossRef]  

13. G. Li, Q. Zhou, G. Xu, et al., “Lidar-radar for underwater target detection using a modulated sub-nanosecond Q-switched laser,” Opt. Laser Technol. 142, 107234 (2021). [CrossRef]  

14. W. Li, S. Guo, Y. Zhai, et al., “Denoising of the multi-slit streak tube imaging LiDAR system using a faster non-local mean method,” Appl. Opt. 60(34), 10520–10528 (2021). [CrossRef]  

15. M. Allgaier, V. Ansari, C. Eigner, et al., “Streak camera imaging of single photons at telecom wavelength,” Appl. Phys. Lett. 112(3), 3 (2018). [CrossRef]  

16. J. Wiersig, C. Gies, F. Jahnke, et al., “Direct observation of correlations between individual photon emission events of a microcavity laser,” Nature 460(7252), 245–249 (2009). [CrossRef]  

17. M. Aßmann, F. Veit, J.-S. Tempel, et al., “Measuring the dynamics of second-order photon correlation functions inside a pulse with picosecond time resolution,” Opt. Express 18(19), 20229–20241 (2010). [CrossRef]  

18. C. Zhang, Y. Wang, Y. Yin, et al., “High precision 3D imaging with timing corrected single photon LiDAR,” Opt. Express 31(15), 24481–24491 (2023). [CrossRef]  

19. J. Rapp, Y. Ma, R. M. A. Dawson, et al., “High-flux single-photon lidar,” Optica 8(1), 30–39 (2021). [CrossRef]  

20. H. Yang, L. Wu, X. Wang, et al., “Signal-to-noise performance analysis of streak tube imaging lidar systems. I. Cascaded model,” Appl. Opt. 51(36), 8825–8835 (2012). [CrossRef]  

21. L. Wu, X. Wang, H. Yang, et al., “Signal-to-noise performance analysis of streak tube imaging lidar systems. II. Theoretical analysis and discussion,” Appl. Opt. 51(36), 8836–8847 (2012). [CrossRef]  

22. G. W. Fraser, J. F. Pearson, and J. E. Lees, “Dark noise in microchannel plate X-ray detectors,” Nucl. Instrum. Methods Phys. Res., Sect. A 254(2), 447–462 (1987). [CrossRef]  

23. M. Konnik and J. Welsh, “High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial,” arXivarXiv:1412.4031 (2014). [CrossRef]  

24. A. D. Gleckler, Y. Zhao, and L. Liu, “Multiple-slit streak tube imaging lidar (MS-STIL) applications,” Laser Radar Technology and Applications V. SPIE. Vol. 4035. (2000). [CrossRef]  

25. Y. Zhang, Y. Zhao, L. Liu, et al., “Improvement of range accuracy of range-gating laser radar using the centroid method,” Appl. Opt. 49(2), 267–271 (2010). [CrossRef]  

26. K. Zhang, R. Fan, X. Li, et al., “FFDNet: Toward a fast and flexible solution for CNN-based image denoising,” IEEE Trans. on Image Process. 27(9), 4608–4622 (2018). [CrossRef]  

27. Z. Chen, Y. Wang, Y. Yin, et al., “Accuracy improvement of imaging lidar based on time-correlated single-photon counting using three laser beams,” Opt. Commun. 429, 175–179 (2018). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1.
Fig. 1. (a) Schematic diagram of the high-frequency laser and few-photos echo streak tube imaging lidar system. The system includes: pulsed laser; beam splitter(BS); beam expander (BE05); diaphragm(D); polarizer(P1,P2); cylinder lens(CL); polarization beam splitter (PBS); objective lens(OL); bandpass filter(BPF); reflector(R);quarter wave plate(QWP); galvanometer mirror(GM), avalanche Photo Diode(APD); CMOS; digital delay generator; sweep circuit; streak tube; signal generator. (b) The working principle schematic of the streak tube. (c) Target echo image recorded by CMOS sensor. (d) The time sequence diagram of CMOS and galvanometer mirror. (e) Physical drawing of the system.
Fig. 2.
Fig. 2. (a) Real detected signals with its gray value variation in the time dimension. (b) The priori system imaging response function obtained by high-energy laser. (c) Time-gray value curve obtained after the cross-correlation algorithm.
Fig. 3.
Fig. 3. The gray value of the system in the time dimension for different η (500, 3000 and 10000).
Fig. 4.
Fig. 4. (a)The temporal distortion image of a planar target by the STIL system. (b)The bending effect obtained through the cross-correlation algorithm and the curve after correction of the planar target
Fig. 5.
Fig. 5. (a) 3D printed resin depth board. (b) Elevation map obtained by cross-correlation algorithm under high-energy laser. (c) Depth image as the reference. (d) Horizontal curve of 0.5mm depth gradient. (e) Vertical curve of 0.5mm depth gradient.
Fig. 6.
Fig. 6. Depth resolution results using different algorithms for acquisition times of 50ms, 300ms and 1000ms respectively.
Fig. 7.
Fig. 7. The range accuracy for the cross-correlation algorithm with acquisition times of 50ms, 300ms, and 1000ms. The first row shows the 300 experiments results of depth measurements. The vertical axis represents the discrete distribution of ROE. The second row shows the histogram statistics for the 300 results, including the central tendency and SD of the ranging data.
Fig. 8.
Fig. 8. (a) The physical image of USAF1951 negative standardized resolution chart. (b) Real reference reflectivity image. (c) Reflectivity images of the resolution plate using different algorithms with different acquisition times.
Fig. 9.
Fig. 9. The horizontal and vertical intensity profiles extracted from the reference real image.
Fig. 10.
Fig. 10. Two natural scenes. (a) Geometric composition. (b) David model.
Fig. 11.
Fig. 11. The depth images of geometric composition and David model. (a) Acquisition time: 50ms. (b) Acquisition time: 300ms. (c) Acquisition time: 1000ms.
Fig. 12.
Fig. 12. The normalized reflectivity images of geometric composition and David model. (a) Acquisition time: 50ms. (b) Acquisition time: 300ms. (c) Acquisition time: 1000ms.
Fig. 13.
Fig. 13. The trend of RMSE in mm for depth and PSNR in dB and SSIM for reflectivity by different algorithms on the geometric combination and David model.

Tables (2)

Tables Icon

Table 1. Summary of the main system parameters.

Tables Icon

Table 2. The results of RMSE in mm for depth and PSNR in dB and SSIM for reflectivity by different algorithms on Geometric combination and David model.

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

η = T . f R F
I t o t l e = i = 1 η I e i + j = 1 η N b j + N d
S N R η = E ( i = 1 η I e i ) 2 E ( j = 1 η N b j + N d ) 2
N d 1 = i = 1 η α P o i s s o n ( I e i ) + j = 1 η g I e j N o r m a l ( 0 , δ P R N U )
N d 2 = i = 1 η β P o i s s o n ( I e i ) + T . D d a r k + N o r m a l ( 0 , δ r e a d )
S N R η = I e ¯ 2 α 2 I e ¯ + g 2 I e ¯ 2 δ P R N U 2 + β 2 I e ¯ + D d a r k 2 f R F 2 + δ r e a d 2 η 2 + N b ¯ 2
S N R η I e ¯ 0 = I e ¯ 2 D d a r k 2 f R F 2 + δ r e a d 2 η 2 + N b ¯ 2
t ( x , y ) = arg max t i = 1 N t b g t + i ( x , y ) × I R F i
S B R = 1 m n i = 1 m j = 1 n t = 1 N t b g i , j ( t ) N t b N o i s e ¯ N t b N o i s e ¯
R M S E ( t , t ^ ) = 1 m n i = 1 m j = 1 n ( t i , j t ^ i , j ) 2
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.