Single-pixel imaging uses a single-pixel detector, rather than a focal plane detector array, to image a scene. It provides advantages for applications such as multi-wavelength, three-dimensional imaging. However, low frame rates have been a major obstacle inhibiting the use of computational ghost imaging technique in wider applications since its invention one decade ago. To address this problem, a computational ghost imaging scheme, which utilizes an LED-based, high-speed illumination module is presented in this work. At 32 × 32 pixel resolution, the proof-of-principle system achieved continuous imaging with 1000 fps frame rate, approximately two orders larger than those of other existing ghost imaging systems. The proposed scheme provides a cost-effective and high-speed imaging technique for dynamic imaging applications.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
As an alternative to using a focal-plane detector array, single-pixel computational ghost imaging , an improved scheme of ghost imaging [2, 3] and closely related to single-pixel imaging [4–6], retrieves a two-dimensional image by recording only the total light intensities in each component of a spatial sampling basis, rather than capturing it with a pixelated array as in a digital camera. Though not performing as well as detector array in conventional visible imaging, computational ghost imaging provides advantages for applications such as multi-wavelength imaging [7–9] and three-dimensional imaging [10–12], both of which pose difficulties for conventional cameras.
Low frame rate has been a major obstacle for computational ghost imaging to develop into a more practical technique. Many works have sought to address this problem and improved the frame rate to over 10 frames per second (fps) in recently years [11–14]. The approaches include using devices such as spatial light modulator (SLM) , and utilizing compressive sensing [15–17] to reduce the number of acquisitions that computational ghost imaging requires to reconstruct an image. However, even the digital-micromirror-device (DMD), the fastest SLM for the time being, has an upper limitation of 22 KHz modulation rate, which leads to ~10 fps at 32 × 32 pixel resolution for fully sampled differential ghost imaging [18, 19]. Compressive sensing does decrease the required number of acquisitions for each frame, however, the subsequent convex optimization procedure to reconstruct the image requires a computational overhead, which poses difficulties for real-time dynamic applications. Furthermore, the problems to be solved by the convex optimization are ill-posed, therefore the solutions could be non-deterministic . Other works aiming to increase the frame rate, including using a quadrant detector  or multi-diode design , are still limited by the modulation rate of the SLM.
In this work, we significantly improved the frame rate using a computational ghost imaging scheme based on a high-speed LED illumination module. With the help of the evolutionary compressive sensing [7, 11], which is a low computational and deterministic reconstruction algorithm, the proof-of-principle system experimentally achieved continuous imaging at a 1000 fps frame rate, which is approximately two orders higher than those of existing computational ghost imaging systems. The presented scheme enables us to capture images of dynamic scenes, which would be otherwise motion-blurred by other existing computational ghost imaging schemes as well as many conventional cameras.
In computational ghost imaging, Fig. 1(a), an SLM creates spatial modulated light field Ei (0) (here i refers to the ith modulation), which propagates distance L through free space and illuminates the object with the light field Ei (L). The total transmitted or reflected intensity Si is then collected and measured by a large area “bucket” detector. More practically, computational ghost imaging is performed with an imaging lens, Fig. 1(b), by which the modulation plane and the object plane are imaged to each other. The intensity distribution is then , where β is the magnification determined by the imaging conjugation. Based on the knowledge of intensity distribution Pi and corresponding measurement Si the reconstruction of the ghost image can be undertaken with various degrees of sophistication [13, 23]. The simplest algorithm weights each Pi by the value of the fluctuation of Si and then sums these weighted Pi 
According to Eq. (1), the time to obtain one ghost image depends on the required acquisition time of N measurements. Early computational ghost imaging schemes [1, 5, 10, 23] used partially correlated patterns to provide modulation. It usually requires a large number of measurements to reconstruct an image, therefore the frame rate of the system is jeopardized. Recent works using orthogonal patterns [8, 11, 16, 25] to provide more efficient modulation, which leads to a smaller N while yielding better image qualities compared to those using partial correlated patterns. However, the inversely proportion relationship still stands, except that now an n × n pixelated scene can be fully sampled with N = n × n orthogonal measurements. Specifically, to obtain images with pixel number N, the system frame rate is Rmod/N, where Rmod is the modulation rate of system as in patterns/s. To further improve the frame rate of the ghost image, one approach is to decrease N by utilizing compressive sensing. Provided the scene is sparse, compressive sensing [15–17] can reconstruct ghost image with N < n × n measurements by under sampling the scene and solving the optimization problem. However, the optimization procedure could lead to a computational overhead and a non-deterministic reconstruction.
In conclusion, if the scene is not sparse and a deterministic reconstruction is desired, then at least N = n × n measurements are required to reconstruct an n × n pixelated scene. Consequently, to increase the frame rate of computational ghost imaging, the rational choice is to increase the modulation rate of the SLM.
3. Illumination module
Besides using SLM, structured illumination can be performed by using electro-optic modulation of broadband laser sources to conduct basis scan and compressive imaging [26–28]. Fast switching LED micro-arrays have been used 3D phase and intensity microscopy, super-resolution microscopy, and neural stimulation [29, 30], and the concept of using LED arrays in the context of computational ghost imaging was recently considered theoretically in . Here, taking advantage of the fact that the switching time of a single LED is less than 1 μs, we developed an LED array module to provide high-speed structured illumination for computational ghost imaging.
In this work, we chose to use the Hadamard basis  as the displayed patterns. There are three reasons for this choice. First, the patterns are orthonormal, therefore provide more efficient scene sampling. Second, the patterns are binary, therefore is easier and faster to be displayed by LEDs than greyscale patterns. Third, all patterns from the Hadamard basis contain a certain symmetry, which enables us to achieve a high pattern rate. More information on utilizing Hadamard matrix in computational ghost imaging or single-pixel imaging can be found in our previous works [15, 33]. The details of developing the LED illumination module are specified as follow.
Module composition. The LED structured illumination module consists of a field-programmable gate arrays (FPGA, Xilinx Spartan XC6SLX9-2FTG256C, 576 Kb RAM and 256 user I/O ports), a drive circuit and an LED array with 32 × 32 LEDs (LDM788AX, wavelength 613 – 630 nm, 2 mm × 2 mm, 10mW).
Display strategy. There are two conventional strategies to display binary patterns on an LED array. The point control strategy, Fig. 2(a), controls each LED with one I/O port individually, meaning n2 I/O ports are required for an n × n resolution LED array. With this strategy, the display rate of the LED array can be as fast as the switching rate of a single LED. However, if the resolution of the LED array is increased, it is difficult for one FPGA to provide enough I/O ports. The line control strategy, Fig. 2(b), controls each row or each column with one port, and uses line scanning technique to display a pattern line by line. It requires only 2n I/O ports for an n × n resolution LED array, therefore avoids the problem of demanding too many I/O ports as resolution goes higher. However, it poses the problem of a much lower display rate because it scans the pattern line by line, and display one pattern in the time of n × LED switching time. Fortunately, for the particular case of Hadamard illumination, we found that all Hadamard patterns contain a symmetry, which enables us to separate each pattern into two parts. Importantly, both parts can be displayed by the line control approach, meaning that by sequentially displaying the two half parts, we can get the structured illumination of one Hadamard pattern in twice the LED switching time, as shown in Fig. 2(c).
Pattern rate. In our module, the controlling signals sent by FPGA to 64 I/O ports were set to 1 MHz because the LED switching time is smaller than 1 μs. Using the display strategy we specified above, the pattern rate of the LED array was 500K patterns/s.
4. Results and discussion
Figure 3 illustrates the LED based computational ghost imaging experiment set-up, which is similar to Fig. 1 except that an LED array has replaced the SLM. The LED array was of 32 × 32 pixel resolution with the array size of 64 mm × 64 mm. The array displayed a preloaded sequence of Hadamard patterns at rate of 500K patterns/s. To remove noise of light fluctuation, differential signals were obtained by displaying each Hadamard pattern, followed by its inverse pattern, and taking the difference in the measured two intensities [15, 20, 34]. A camera lens (Nikon AF Nikkor, f = 50 mm, F/1.4D) was located 250 mm from the array and imaged the Hadamard patterns to 12 mm × 12 mm structured illumination on a rotating object. The object was a black disk uniformly carved with ten numbers from ‘0’ to ‘9’ and it rotated at the specified speed. The transmitting light intensities corresponding to different Hadamard patterns were collected by a singlet lens (BOCIC, OCL3-050-050, f = 50 mm, F/1) and measured by a single-pixel detector (Thorlabs PDA100A-EC, 320 nm - 1100 nm, active area ϕ10 mm, gain 20 dB). A high dynamic range digitizer (PicoScope 6404D, sampling at 2 GS/s), synchronized with the LED array, acquired and transferred the intensity data to a computer for image reconstruction.
To fully sample the object and reconstruct a 32 × 32 pixel image, 2048 patterns, 1024 Hadamard patterns and their inverse, were displayed. By utilizing evolutionary compressive sensing , it is also possible to reconstruct images with less patterns at a higher frame rate. In short, the evolutionary compressive algorithm chooses a subset of the Hadamard patterns to sample the scene, by selecting the patterns with the most significant intensities measured in the previous frame. To keep its robustness to dynamics, a fraction of randomly selected patterns which were not used previously were also selected. Figure 4(a) shows three reconstructed images of the static object by using different numbers of patterns. The image size was 12 mm × 12 mm with 32 × 32 pixel resolution, the same as that of the structured illumination. The times to display 2048, 1024 and 512 patterns were 4.1μs, 2.0μs and 1.0μs respectively. The image quality at 1000 fps was only mildly degraded when the image was reconstructed with 75% fewer patterns because evolutionary compressive sensing only samples the sub set of patterns having large weights. Noting here that the 1000 fps was achieved by using a 25% compressive rate, if full sampling was performed, the actual frame rate would be 250 fps. On the contrary, when the disc rotated at 8 rounds/s, i.e. the object moved at a speed of 1.51 m/s, at the tangential direction of disc circumference, Fig. 4(b) shows that the image qualities at 250 fps and 500 fps were compromised correspondingly by motion blur while the image quality at 1000 fps remained similar to that in the static case. The signal-to-noise ratio (SNR) of the images were obtained using red solid rectangle feature as signal and red dash rectangle background as noise. The values of the image SNR were in a good agreement with the observation above. It is worth mentioning that motion blur in computational ghost imaging is different from that in conventional imaging, though both lead to image quality degrading. The motion blur is related to the frame rate, the object size and the object moving velocity, detailed discussion can be found in Supplementary Note 3 of .
Figure 5(a) shows three images of the object at different rotational speeds. Because of the high frame rate, the image quality was still maintained as the object rotated faster. As an illustrative comparison, Fig. 5(b) shows three images captured by a commercial camera (Nikon D5300, with Nikon AF Nikkor lens, f = 50 mm, F/1.4D) operating at its highest frame rate (50 fps), where the motion blur became more severe as the object rotated faster. In the instance of Nikon D5300 camera, it is possible for it to get rid of the motion blur by taking a single shot with a 0.001 s shutter time (equivalent to 1000 fps). However, it is much more difficult for the camera to operating continuously at this frame rate. Visualization 1 shows an illustrative comparison of the video recorded by the conventional camera at 50 fps and the one obtained by our system at 1000 fps. We realize that there are expensive scientific cameras on the market which could reach higher frame rates with a higher pixel resolution, the video here only provided a visualization of the motion blur caused by low frame rates. Because commercial cameras perform calibration and pre-processing on the raw image data, we also performed Gaussian smoothing and thresholding on the data obtained by our system in Fig. 5 and Visualization 1, therefore these images didn’t contain the background noise exhibited in Fig. 4. Although it might be unnecessary for human operator to see these images at 1000 fps, it could be crucial for machine vision to have the ability of capturing images of fast objects. As a matter of fact, the switching time of a single LED can be reduced to several nanoseconds, therefore the illumination rate could be further increased and potentially reach 100K fps.
Illumination energy is an issue in the proposed system. Large drive currents could be applied to provide enough illumination power for long distance imaging application. However, the dense LED array would cause a severe heating problem. In this experiment, the current was limited to 2A. Given its relatively weak illumination but high frame rate, the imaging system we proposed here might be more suitable for living cell microscopy techniques, such as structured illumination microscopy, whose frame rate is still under 100 Hz [35, 36].
In this work, we presented a computational ghost imaging scheme based on a high-speed LED illumination module. Taking advantage of the fact that the switching time of a single LED is smaller than 1 μs, and utilizing the symmetry presented in Hadamard patterns, we were able to develop a 32 × 32 LED array module which could illuminate a scene with 500 KHz pattern rate. The proof-of-principle system experimentally achieved 32 × 32 resolution imaging at 1000 fps frame rate, which is approximately two orders higher than that of existing computational ghost imaging systems. The presented scheme enables us to capture images of dynamic scene, which would be otherwise motion-blurred by other ghost imaging schemes as well as many commercial cameras. With the potential of reaching even higher frame rates, the LED array based computational ghost imaging system could be a high-speed, cost-effective imaging solution for living microscopy and many other applications.
Natural Science Foundation of Beijing Municipality (4172039); National Natural Science Foundation of China (61675016, 61307021); UK Quantum Technology Hub in Quantum Enhanced Imaging (EP/M01326X/1).
The authors declare that there are no conflicts of interest related to this article.
References and links
1. J. H. Shapiro, “Computational ghost imaging,” Phys. Rev. A 78(6), 061802 (2008). [CrossRef]
4. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. Kelly, and R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE Signal Process. Mag. 25(2), 83–91 (2008). [CrossRef]
5. Y. Bromberg, O. Katz, and S. Silberberg, “Ghost imaging with a single detector,” Phys. Rev. A 79(5), 053840 (2009). [CrossRef]
7. N. Radwell, K. J. Mitchell, G. M. Gibson, M. P. Edgar, R. Bowman, and M. J. Padgett, “Single-pixel infrared and visible microscope,” Optica 1(5), 285–289 (2014). [CrossRef]
8. M. P. Edgar, G. M. Gibson, R. W. Bowman, B. Sun, N. Radwell, K. J. Mitchell, S. S. Welsh, and M. J. Padgett, “Simultaneous real-time visible and infrared video with single-pixel detectors,” Sci. Rep. 5(1), 10669 (2015). [CrossRef] [PubMed]
10. B. Sun, M. P. Edgar, R. Bowman, L. E. Vittert, S. Welsh, A. Bowman, and M. J. Padgett, “3D computational imaging with single-pixel detectors,” Science 340(6134), 844–847 (2013). [CrossRef] [PubMed]
11. M.-J. Sun, M. P. Edgar, G. M. Gibson, B. Sun, N. Radwell, R. Lamb, and M. J. Padgett, “Single-pixel three-dimensional imaging with time-based depth resolution,” Nat. Commun. 7, 12010 (2016). [CrossRef] [PubMed]
13. B. Lochocki, A. Gambín, S. Manzanera, E. Irles, E. Tajahuerce, J. Lancis, and P. Artal, “Single pixel camera ophthalmoscope,” Optica 3(10), 1056–1059 (2016). [CrossRef]
15. E. J. Candes, “Compressive sampling,” in Proceedings of the International Congress of Mathematicians (International Mathematical Union, 2006), pp. 1433–1452.
16. D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). [CrossRef]
18. M.-J. Sun, M. P. Edgar, D. B. Phillips, G. M. Gibson, and M. J. Padgett, “Improving the signal-to-noise ratio of single-pixel imaging using digital microscanning,” Opt. Express 24(10), 10476–10485 (2016). [CrossRef] [PubMed]
19. D. B. Phillips, M.-J. Sun, J. M. Taylor, M. P. Edgar, S. M. Barnett, G. M. Gibson, and M. J. Padgett, “Adaptive foveated single-pixel imaging with dynamic supersampling,” Sci. Adv. 3(4), e1601782 (2017). [CrossRef] [PubMed]
20. E. J. Candès and M. B. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag. 25(2), 21–30 (2008). [CrossRef]
21. M.-J. Sun, W. Chen, T.-F. Liu, and L.-J. Li, “Image retrieval in spatial and temporal domains with a quadrant detector,” IEEE Photonics J. 9(5), 3901206 (2017). [CrossRef]
22. M. Herman, J. Tidman, D. Hewitt, T. Weston, and L. McMackin, “A higher-speed compressive sensing camera through multi-diode design,” Proc. SPIE 8717, 871706 (2013). [CrossRef]
25. W. Le and S. Zhao, “Fast reconstructed and high-quality ghost imaging with fast Walsh–Hadamard transform,” Photon. Res. 4(6), 240–244 (2016). [CrossRef]
26. M. Alemohammad, J. R. Stroud, B. T. Bosworth, and M. A. Foster, “High-speed all-optical Haar wavelet transform for real-time image compression,” Opt. Express 25(9), 9802–9811 (2017). [CrossRef] [PubMed]
27. B. T. Bosworth, J. R. Stroud, D. N. Tran, T. D. Tran, S. Chin, and M. A. Foster, “High-speed flow microscopy using compressed sensing with ultrafast laser pulses,” Opt. Express 23(8), 10521–10532 (2015). [CrossRef] [PubMed]
28. Q. Guo, H. Chen, Z. Weng, M. Chen, S. Yang, and S. Xie, “Compressive sensing based high-speed time-stretch optical microscopy for two-dimensional image acquisition,” Opt. Express 23(23), 29639–29646 (2015). [CrossRef] [PubMed]
29. L. Tian and L. Waller, “3D intensity and phase imaging from light field measurements in an LED array microscope,” Optica 2(2), 104–111 (2015). [CrossRef]
30. N. Grossman, V. Poher, M. S. Grubb, G. T. Kennedy, K. Nikolic, B. McGovern, R. Berlinguer Palmini, Z. Gong, E. M. Drakakis, M. A. Neil, M. D. Dawson, J. Burrone, and P. Degenaar, “Multi-site optical excitation using ChR2 and micro-LED array,” J. Neural Eng. 7(1), 16004 (2010). [CrossRef] [PubMed]
31. S. Onose, M. Takahashi, Y. Mizutani, T. Yasui, and H. Yamamoto, “Single Pixel Imaging with a High-Frame-Rate LED Array,” in JSAP-OSA Joint Symposia 2016 Abstracts (Optical Society of America, 2016), paper 13a_C301_10.
32. W. K. Pratt, J. Kane, and H. C. Andrews, “Hadamard transform image coding,” Proc. IEEE 57(1), 58–68 (1969). [CrossRef]
33. M.-J. Sun, L.-T. Meng, M. P. Edgar, M. J. Padgett, and N. Radwell, “A Russian Dolls ordering of the Hadamard basis for compressive single-pixel imaging,” Sci. Rep. 7(1), 3464 (2017). [CrossRef] [PubMed]
35. P. Kner, B. B. Chhun, E. R. Griffis, L. Winoto, and M. G. Gustafsson, “Super-resolution video microscopy of live cells by structured illumination,” Nat. Methods 6(5), 339–342 (2009). [CrossRef] [PubMed]