Ghost imaging system requires a large number of samples to reconstruct the object. Computational ghost imaging can use well-designed pre-modulated orthogonal patterns to reduce the requirement of sampling number and increase the imaging quality, while the rotating ground glass (RGG) scheme cannot. Instead of the pre-modulation method, a post-processing method using Gram-Schmidt process to orthonormalize the patterns in a RGG scheme is introduced. Reconstructed ghost image after the Gram-Schmidt process (SGI) are tested using the quality indicators such as the Contrast-to-Noise (CNR), the Peak Signal to Noise Ratio (PSNR), the Correlation Coefficient (CC) and reducing the Mean Square Error (MSE). Simulation results show that this method has obvious advantage on enhancing the efficiency of image acquisition, and the sampling number requirement drops from several thousands to a few hundreds in ideal condition. However, in actual system with noise, the image quality from SGI declines in large sampling number, for noise and errors accumulate in the orthonormalization process. So an improved Group SGI method is then developed to avoid this error accumulation, which behaves effectively in reconstructing the image from experimental data and show good performances in large sampling number too. Since this method do not change the relationship between the reference patterns and the bucket values, it can easily combine with most of reconstruction algorithms and improve their reconstruction efficiency.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Ghost imaging (GI) is a kind of indirect imaging scheme which images an object nonlocally [1,2]. GI schemes have shown advantages in imaging lidar , three-dimensional imaging [4,5] and photon-limited imaging . The most notable feature of GI is that the object is reconstructed from a large number of measurements on two quantities: light patterns that never reach the object and total intensities of the light patterns which transmitted through or reflected from the object. The light patterns that contain no information of the object are called the reference patterns, and the light whose intensities are detected by a “bucket” detector with no spatial resolution is the signal. In 1995, Pittman et al. realized the first ghost imaging experiment in laboratory environment by entangled photon pairs . After that, a variety of experimental forms gradually emerged. Besides entangled photon pairs, light sources can also be thermal light from hollow-cathode lamp , sunlight after atomic optical filter , pseudo-thermal light generated by a rotating ground glass (RGG) , computational light patterns produced by Spatial Light Modulator (SLM) or Digital Micro-mirror Device (DMD) [10–12], and even X-ray [13,14] and atoms .
Generally speaking, the reference pattern of GI mainly has three different types. The first is point-by-point scanning, such as entangled photon pairs. The second is random patterns, such as thermal and pseudo-thermal light. The last is well designed patterns, mainly based on computational GI. For the well designed patterns, the reconstruction time of the GI system can be much shorter and the quality of imaging be much higher [16–18]. However, these methods always need a SLM or DMD which has a low damage threshold and is not suitable for the long distance sensing. Ghost imaging with pseudo-thermal light generated by a RGG presents more potential in practical applications, particularly in remote sensing, for the RGG is able to modulate quite high power laser. Many properties of this scheme are investigated theoretically and experimentally [19–23]. However, as the reference patterns from a RGG are random modulated, there is a high constant background that reduces the reconstructing efficiency and the image quality . Lots of studies focus on enhancing the visibility of the reconstructed image in the GI system with random patterns by a RGG [24–27]. While according to our knowledge, a method that changes the detected random patterns modulated by a RGG into orthonormal patterns has not been proposed yet, which can greatly improve the reconstructing efficiency and the image quality.
In this paper, we introduce a data preprocessing method to improve the reconstructing process in a GI system with pseudo-thermal light. We orthonormalize the reference patterns by Gram-Schmidt process into new orthonormal patterns, and the image can be reconstructed then. This method is named Schmidt GI (SGI). We run a simulation in ideal condition to test the feasibility of this method, and evaluate the performance under different noise levels. Then we use the SGI method to reconstruct the image by using the experimental data. Results show that our method can effectively reduce the correlation between different measurements, the sampling number requirement and improve the quality of the reconstructed image.
2. Theoretical model
This orthonormalization method can be described based on a typical pseudo-thermal light ghost imaging system, as shown in Fig. 1, where Φm is the mth matrices of reference patterns, and ym is the mth measured values of bucket detector. Supposing that the two arms are of equal length, the mth measurement of the two detectors satisfyEq. (1) can be expressed as a problem of solving systems of linear equation 28], the object can be reconstructed by
Equations (6) and (7) indicate that as we get a new measurement, we subtract all the projection components of the previous orthogonal patterns on this pattern. After this operation, the new set of R̃1, R̃2, · · · , R̃N is orthogonal. As the number of measurements increases, more and more components of patterns are subtracted and the intensity of patterns gets smaller. We assume that the orthogonal patterns provide similar amount of information of the object, so the patterns should be normalized then. The new orthonormal basis R̃′1, R̃′2, · · · , R̃′N can be obtained by dividing the orthogonal patterns’ own 2-norm, as
In order to maintain the relationship between the patterns and buckets, we do some similar operations for Y to generate the new Ỹ′, whose elements ỹ′ are calculated by
3. Method and simulation results
To test the feasibility of the SGI method, we run a simulation in ideal condition. As shown in Fig. 2, we use a CMOS camera (Thorlabs DCC 3240C) to capture reference patterns, which are Φ1, Φ2, ..., ΦN. To create an ideal condition to run the simulation, we use these random patterns and a binary picture mask generated by software to calculate bucket values y1, y2, ..., yN according to the Eq. (1). The object is a binary picture of two letters “GI” with the size of 140 × 140 pixels, so are the reference patterns. We reshape the patterns to row vectors R1, R2, ..., RN, each of whose length is 19600. Processes of orthogonalization and normalization introduced in section 2 are carried out on the patterns and buckets. R̃′1, R̃′2, ..., R̃′N and ỹ′1, ỹ′2, ..., ỹ′N are generated.
The matrix A and Ã′ can be use to predict the property of GI . We separately calculate correlation coefficients (CC) between row vectors of each matrix and between column row vectors of each. Figure 3 shows the changes in CC between row vectors of A and those of Ã′. In Fig. 3(a), besides for most of the values in the range of −0.2 to 0.2, there are still a large number of different measurements whose CC are within the range 0.6 to 1. They have strong relativity, so most of their contributions are redundant. This phenomenon will cause time wasting in correlation operations. As shown in Figs. 3(c) and 3(d), the distribution change of the correlation coefficients, the variances and the sum of CC both reduce significantly after the processing. In Fig. 3(d), almost all values appear in the vicinity of zero, that is to say, all of the new reference patterns provide totally new information. Thus, the image can be reconstructed by much smaller sampling number.
In the other hand, orthonormalising the row vectors of the matrix A also affects the CC between column vectors, although we do not operate on the column vectors directly. Every column vector in A represents light intensities on a single pixel in the measurements. The spatial coherence of pseudo-thermal light field determines that these column vectors are not completely independent. Instead, the whole pattern is composed of speckle particles with a certain size, which makes the reconstructed image blur. Figure 4 shows the changes in correlation coefficients between column vectors before and after the orthonormalization process. Limited by the computer ability, we only calculate the CC between the first 4200 column vectors rather than all the 19600 column vectors, and these are enough to find the changes. The width of correlation area of pixels in space and the distribution width of CC between pixels are both narrowed after the process on the row vectors, or on the reference patterns. Changes of correlation coefficients between row vectors and between column row vectors indicate that the SGI method effectively reduces the correlation noise between different row and column vectors in the measure matrix, which leads to faster reconstruction and better imaging quality.
The new R̃′1, R̃′2, ..., R̃′N and ỹ′1, ỹ′2, ..., ỹ′N are used to recover the object information with the traditional correlation function of intensity fluctuation according to Eq. (10). In the next analysis, we will present the results of SGI, compressive ghost imaging (CGI), and directly using R1, R2, ..., RN and y1, y2, ..., yN to do correlation of fluctuation as GI. We compare the images from the three kinds of methods, and the results (which are reshaped back to 140 ×140) under different measurements are shown in Fig. 5.
According to Fig. 5, we find that the SGI has a great reconstructing efficiency, and the “GI” letters can almost be identified when the measurement number m = 100. It is fairly small compared with those in most GI systems, which are usually thousands of times. When the number of measurements m = 500, the image by SGI looks the same as that by CGI with 5000 measurements already. When the number of measurements m = 5000, the two letters image by SGI has a clearer and sharper outline than those by GI and CGI, and the filling inside the edge is more smooth. The object in this simulation is a a binary picture, and it is clear that the image reconstructed by SGI is more similar to the object.
In order to better evaluate the performance of different methods rather than intuitively judgments, we adopt four evaluating indicators of image quality: the Contrast-to-Noise (CNR) [25,30,31], the Mean Square Error (MSE), the Peak Signal to Noise Ratio (PSNR)  and the Correlation Coefficient (CC). They can be calculated by
The performance of the reconstructed images by different methods are shown in Fig. 6. Before m = 1000, as the measurements number increases, the image quality by all methods is improved quickly (The MSE value is the lower the better), while the growth rate by the SGI method is the fastest. If more measurements are taken, the image quality by GI and CGI tend to be saturated. The case for SGI is quite different, all the indicators are still improving. These trends agree with our previous analysis. As the measurement number increases, the new measured pattern contains more and more redundant components which also exist in previous patterns, and the new component with new information is smaller and smaller. So measurements contribute less and less to the image reconstructing. However, the SGI method subtracts all the projection components of the previous patterns and amplifies the new component to a same level as the previous patterns, so the image quality keeps improving even in a large measurement number. Generally speaking, the performance of SGI has a higher rate of growth and higher convergent value.
4. Noise analysis and experimental results
In the previous analysis in ideal condition, practical noises and errors are not included, such as the background noise, the quantization error and thermal noise of detectors. And sometimes, the distance of two arms is not equal, thus the light field illustrates on the object and CMOS camera are biased by diffraction. To find the influence of such noise and errors, we run a simulation under different noise levels.
We simply assume that all the noise and errors are equivalent to an additive white Gaussian noise (AWGN) on the bucket values, and the signal to noise ratio (SNR) Q can thus be calculated byFig. 7. It is clear that the image qualities still increase faster by SGI method than by GI method, but only in a certain sampling number range. The image qualities get bad when the measurement number is out of that range. Taking CNR for example, as shown in Fig. 7(a), when Q = 60dB, there is an extreme point for the line at about m = 360. When Q = 50dB, this point change to another measurement number (about m = 240), and the maximum CNR also reduce. When Q = 40dB, this point exists at about m = 160 and the CNR here is even smaller. It is quite different from the trends of GI, which almost has no change between conditions of Q = 40dB and Q is ideal.
The noise level is not the only factor that affects the best sampling number. We do another simulation by using three more different images, an anti GI, a cat and a hawk, as shown in Fig. 8(a). The anti GI is an orthogonal image of the original GI, where “1” and “0” are exchanged. The cat and hawk both have the same size as the GI, which means the counts of “1” pixels are roughly the same. They all contain about 2500 “1” pixels, which account for 12.8 percent of the total 140 × 140 image. The quality lines of the four images by SGI in same noise levels are compared, and the optimal sampling numbers via noise levels are shown in Fig. 8(b)–8(c). We can find that, although the optimal sampling numbers of GI and anti GI are different in the same noise level, the optimal sampling numbers of the cat and the hawk images, which are quite different but have the same size as GI, are almost the same. It indicates that the object size also affects the optimal sampling number when the noise level is fixed. It is not a good feature for common imaging, as the object to be shoot can be quite different. But for many imaging applications that we know a general size of the object, just like character recognition systems, we can use a known object to calibrate the system and find the optimal sampling number, in the range of which the image quality can increase fast.
We also test the SGI method in an experiment, and the experimental setup is shown in Fig. 1. The reference patterns are divided into two parts. One of them is captured by the camera. The other transmitted through an object with two letters “GI” and be detected by a bucket detector. They are no longer be calculated in this section. We measure 20000 patterns and divide them into 4 teams, each of which contains 5000 measurements. As the SGI method is noise-sensitive, we take 1000 measurements on the bucket value for each reference pattern and record them all in order to enhance the detection precision and reduce the influence of random error. We take each first measurement on the bucket value for each reference pattern, implement the SGI processing and get reconstructed images as the SGI results. We also take the average value of the 1000 measurements on the bucket value for each reference pattern, perform the same operation and get the SGIavg results. The image qualities, CNR, MSE, PSNR and CC, via the measurement number by using SGI, SGIavg, GI and CGI are shown in Fig. 9. The standard object we used here to calculate the qualities is the binary oversampled CGI result, and the error bars are from the results of the divided 4 teams.
The trend of curves obtained from experimental results are almost the same as those of the above simulations. For CGI and GI, their imaging qualities increase with the number of measurements. For SGI and SGIavg, imaging quality increases faster at small measurement number and drops at large one. Compared with the performance of SGI, the SGIavg results are a little better, because the averaging process reduce the influence of random error in the buckets sampling. Now both the simulations and the experimental results make it clear that the SGI method is easily affected by the noise.
In the process of orthogonalization, new values need to be calculated by subtracting all the previous results. As the operation goes on, the error accumulates quickly. In view of that the error is not a constant but a increasing value, its impact on the system is nonlinear enhancement. Therefore, when the number of measurements is small, the error can be ignored, the image information can be got rapidly by the SGI method. But as the measurement number increases, the speed of the error accumulating is fast, and the quality of the image reconstructed by these data decreases rapidly, so there exists extreme points, in the left of which, the performance from SGI is superior to others. In other words, the SGI only has a certain advantage in small sampling number cases.
5. Group SGI method
To improve the performance of SGI method in large sampling number, we further introduce a Group SGI method. In Fig. 9, the extreme point of SGI in this experiment is at about 200 measurement. As the SGI method shows advantage in this sampling number, we divided the total measurements into several groups, each of which only contains 200 measurements, and we do SGI processing in each group. For the amount of each group is small, the accumulated detection error is not big, and all the measurements can contribute to the image reconstructing.
With the same experimental data in section 4, we use the modified group SGI method to calculate ghost image. The corresponding results are shown in Fig. 10. It is clear that the image qualities by the group SGI method no longer drop under large measurement numbers, and the CNR, MSE, PSNR all indicate that the group SGI method not only reconstructs the image faster, but also has a better final image quality than GI and CGI results, as shown in Figs. 10(a)–10(c). However, the indicator CC shows a different result. In Fig. 10(d), the CC value by CGI is a little better than that by SGI. A probable reason is that the CGI method itself is quite effective in CC reconstructing and we use the binary oversampled CGI result as the standard object, so the CC by group SGI is difficult to compare with that from CGI.
The group size is a key factor that determines the final performance of Group SGI, and it is affected by the noise level and the object size. In actual GI system, we can collect data before actual imaging to estimate the system noise level, and we can use a known object to calibrate the system and find the optimal sampling number, based on which, unknown objects which have similar size with the calibration object can be well reconstructed by the Group SGI method.
It is worth mentioning that group SGI method also has less computation than the SGI. For N measurements, it takes t = N(N − 1)/2 operations for subtracting projection components. However, as we divide these measurements into d groups, it only takes t′ = N(N − d)/(2d) computations. Generally speaking, the computation of group SGI method is d times smaller.
Correlation of the reference patterns greatly restricts the time and quality of ghost imaging, and the non-orthogonality between different patterns and different pixels in each pattern is the main reason for this phenomenon. Gram-Schmidt process can directly make different patterns orthogonal to improve the reconstructing efficiency and enhance the independence of space pixels, and hence reduce the required sampling number and improve the image quality. Simulation results show that, in an ideal environment, data processing in this way has much better performance than CGI and GI, and the required sampling number drops from thousands of times to only a few hundreds.
However, this method is sensitive to noise. The errors induced by noise accumulate at large measurement number. Therefore, the orthogonality of the light field and the robustness of the system need a balance point. One effective way is to divide the original data into several groups and do the orthonormalization for each individual group, and use data from all the groups to reconstruct the object. This group SGI method has better robustness and less computation request than the original SGI method, and presents more potential in practical applications. In fact, this method can also be used in most of the other reconstructing methods as a data preprocessing step and improve their reconstructing efficiency and final performance.
National Natural Science Foundation of China (61631014, 61801042, 61771067, 61401036, 61531003); National Science Fund for Distinguished Young Scholars of China (61225003); Youth Research and Innovation Program of BUPT (2017RC10, 2015RC12).
1. T. B. Pittman, Y. H. Shih, D. V. Strekalov, and A. V. Sergienko, “Optical imaging by means of two photon quantum entanglement,” Phys. Rev. A 52, R3429 (1995). [CrossRef]
2. A. Valencia, G. Scarcelli, M. D’Angelo, and Y. Shih, “Two-photon imaging with thermal light,” Phys. Rev. L 94, 063601 (2005). [CrossRef]
3. C. Zhao, W. Gong, M. Chen, E. Li, H. Wang, W. Xu, and S. Han, “Ghost imaging lidar via sparsity constraints,” Appl. Phys. Lett. 101, 141123 (2012). [CrossRef]
5. M. J. Sun, M. P. Edgar, G. M. Gibson, B. Sun, N. Radwell, R. Lamb, and M. J. Padgett, “Single-pixel three-dimensional imaging with time-based depth resolution,” Nat. Commun. 7, 12010 (2016). [CrossRef] [PubMed]
9. A. Valencia, G. Scarcelli, M. D’Angelo, and Y. Shih, “Two-photon imaging with thermal light,” Phys. Rev. L 94, 063601 (2005). [CrossRef]
10. J. H. Shapiro, “Computational ghost imaging,” Phys. Rev. A 78, 061802 (2008). [CrossRef]
11. Y. Bromberg, O. Katz, and Y. Silberberg, “Ghost imaging with a single detector,” Phys. Rev. A 79, 053840 (2009). [CrossRef]
12. L. Wang and S. Zhao, “Fast reconstructed and high-quality ghost imaging with fast Walsh–Hadamard transform,” Photon. Res. 4(6), 240–244 (2016). [CrossRef]
13. H. Yu, R. Lu, S. Han, H. Xie, G. Du, T. Xiao, and D. Zhu, “Fourier-Transform Ghost Imaging with Hard X Rays,” Phys. Rev. L 117, 113901 (2016). [CrossRef]
18. X. Xu, E. Li, X. Shen, and S. Han, “Optimization of speckle patterns in ghost imaging via sparse constraints by mutual coherence minimization,” Chin. Opt. Lett. 13(7), 071101 (2015). [CrossRef]
21. W. Gong, “High-resolution pseudo-inverse ghost imaging,” Photon. Res. 3(5), 234–237 (2015). [CrossRef]
22. X. Li, C. Deng, M. Chen, W. Gong, and S. Han, “Ghost imaging for an axially moving target with an unknown constant speed,” Photon. Res. 3(4), 153–157 (2015). [CrossRef]
23. J. Li, B. Luo, D. Yang, L. Yin, G. Wu, and H. Guo, “Negative exponential behavior of image mutual information for pseudo-thermal light ghost imaging: observation, modeling, and verification,” Chinese Sci. Bull. 62, 717–723 (2017).
24. D. Z. Cao, J. Xiong, S. H. Zhang, L. F. Lin, L. Gao, and K. Wang, “Enhancing visibility and resolution in Nth-order intensity correlation of thermal light,” Appl. Phys. Lett. 92(20), 201102 (2008). [CrossRef]
25. K. W. C. Chan, M. N. O’Sullivan, and R. W. Boyd, “Optimization of thermal ghost imaging: high-order correlations vs. background subtraction,” Opt. Express 18(6), 5562–5573 (2010) [CrossRef] [PubMed]
27. O. Katz, Y. Bromberg, and Y. Silberberg, “Compressive ghost imaging,” Appl. Phys. Lett. 95, 131110 (2009). [CrossRef]
28. W. Gong and S. Han, “A method to improve the visibility of ghost images obtained by thermal light,” Phys. Lett. A 374, 1005–1008 (2010). [CrossRef]
29. C. Wang, W. Gong, X. Shao, and S. Han, “The influence of the property of random coded patterns on fluctuation-correlation ghost imaging,” J. Opt. 18, 065703 (2016). [CrossRef]
30. P. Zerom, Z. Shi, M. N. O’Sullivan, K. W. C. Chan, M. Krogstad, J. H. Shapiro, and R. W. Boyd, “Thermal ghost imaging with averaged speckle patterns,” Phys. Rev. A 86, 063817 (2012). [CrossRef]