1. Introduction

Compared with conventional microscope, lensless in-line holographic microscope (LIHM) provides a promising alternative microscopic imaging solution because of its advantages of low cost, compactness, wide field-of-view, multilayer imaging capability, and flexible imaging distances [1–5]. In many application areas, e.g., biology and medicine, multispectral or color microscopic images are very useful because of they provide richer information compared with monochromatic images [6,7]. Intuitively, color imaging in LIHM can be achieved by either using multiple laser sources or light-emitting diodes (LEDs) with different color [8–10] or white light sources with different spectral filters [11]. However, the use of multiple light sources or different spectral filters increases the system complexity and cost as well as the image acquisition time because multiple holograms in different color have to be acquired sequentially to obtain the full color image.

To simplify the system design and reduce the cost, we propose a method that can acquire color images in LIHM by using white light illumination without any spectral filters and acquiring only one in-line hologram with a color CMOS imaging sensor. In this case, the in-line color hologram can be decomposed into red (R), green (G), and blue (B) sub-holograms which can then be used to reconstruct the corresponding color components of the sample image. However, because of the coherence requirements in holographic reconstruction, the issues of insufficient spatial and temporal coherence must be resolved so that the sample images of different colors can be reasonably reconstructed. For the issue of insufficient spatial coherence, the existing methods include using a pinhole [12] or a deconvolution process [13]. In our experiment, we chose to use sunlight illumination and the spatial coherence was good enough because the sun is very far away from the earth and can be considered as a point source. For the issue of insufficient temporal coherence, spectral filters can be used with the trade-off of increasing system cost and reducing imaging speed in color imaging. Alternatively, a computational spectral demultiplexing method can be applied [14] with the cost of additional computation time and possibly unreliable convergence by using the iterative phase retrieval algorithm. Previously we also proposed a differential holographic reconstruction method to solve the insufficient temporal coherence issue [15]. However, this method is more useful for ultra-broadband light sources and not so effective for the narrower bandwidths of different color components in the color imaging case. In this paper, we propose to use a deconvolution method to solve the insufficient temporal coherence issue with a synthetic point spread function (PSF) computed from the spectral distribution of each color component. We will show that the deconvolution method is applicable for weakly-scattered amplitude objects.

Compared with traditional color holographic imaging method, our method can significantly reduce the system cost and complexity by using sunlight illumination instead of multiple light source illuminations or white light illumination with different spectral filters.

2. Principles

The schematic of our color LIHM system setup is shown in Fig. 1(a)

 

Fig. 1 (a) Schematic of the color LIHM system setup with sunlight illumination; (b) The spectral response of the red (R), green (G), and blue (B) channels in our color CMOS imaging sensor, which includes the Bayer pattern of RGB filter array; (c) Schematic of decomposing the color RGB hologram into red, green and blue components.

Download Full Size | PDF

For the derivation below, we will use the Cartesian coordinates where the z-direction is along the sunlight propagation direction and the xy-plane is the transverse plane, as shown in Fig. 1(a). And we set the sample plane as z = 0. For weakly-scattered amplitude object, the optical field after the sample plane can be written as 1 + o(x, y; λ), where o(x, y; λ) << 1 is the amplitude transmission coefficient of the sample for light wavelength λ and is real. Here the z coordinate is omitted for simplification. Then the in-line hologram intensity under monochromatic light illumination with wavelength λ can be written as [16], knowing that o(x, y; λ) is real,

For each color channel with spectral response S_X(λ), the sub-hologram can be regarded as the incoherent superposition of holograms with monochromatic light illuminations within the spectrum. In addition, we assume that o(x, y; λ) = o(x, y; X) (X = R, G, or B) is independent of λ within the spectrum S_X(λ) because of the relatively narrow bandwidth for each color channel. Thus, the intensities of the sub-holograms in our color LIHM can be written as:

3. Results

Numerical simulation was first performed to verify the resolution enhancement capability of our deconvolution method with a U. S. Air Force (USAF) target as the sample. In the simulation, the illumination light spectrum was the solar radiation spectrum at sea level, as shown in Fig. 1(b). The pixel size we used in numerical simulation was 2.2 μm, which was identical to that of the imaging sensor used in our experiment. The distance between the sample and the sensor was set as 1.6 mm. The synthetic PSF of each color channel was numerically computed according to Eq. (4), knowing the illumination light spectrum and the spectral response of the Bayer pattern color filter array provided in the datasheet of the imaging sensor.

The simulation result is shown in Fig. 2

 

Fig. 2 Numerical simulation results of image reconstructions in color LIHM under the sunlight illumination.(a)(b)(c) The simulated sub-holograms and the corresponding spectral distributions of the colors red, blue, and green; (d)(e)(f) direct reconstruction results of (a)(b)(c) with center wavelengths of 604 nm, 537 nm, and 455 nm, respectively; (g)(h)(i)reconstruction results of (a)(b)(c) with our proposed deconvolution method; (j) the RGB color image after combining the RGB components in (d)(e)(f); (k) the RGB color image after combining the RGB components in (g)(h)(i). The size of images are 220 × 220 μm (100 × 100 pixels)

Download Full Size | PDF

We then did experiment with a USAF target (R1DS1P, Thorlabs Co., USA) as the sample to further verify the effectiveness of our proposed method. To avoid saturation of the sensor because of strong sunlight during the experiment, a neutral-density filter with optical density (OD) of 2 was introduced. Notice that the exact OD of the filter is not critical as long as the imaging sensor is not saturated. And it’s possible to remove the filter if other white light sources with adjustable intensity are used. The synthetic PSF used in the experiment was the same as in the numerical simulation. The distance between the sample and the sensor was around 1.6 mm. The exposure time of the imaging sensor in our experiment was 0.2 ms.

The experimental result is shown in Fig. 3

 

Fig. 3 Imaging results of the USAF target by our color LIHM system under sunlight illumination. (a)(b)(c) The sub-holograms and the corresponding spectral distributions of the color red, blue, and green. (d)(e)(f) direct reconstruction results of (a)(b)(c) with center wavelengths of 604 nm, 537 nm, and 455 nm, respectively; (g)(h)(i) reconstruction results of (a)(b)(c) with our proposed deconvolution method; (j) the RGB color image after combining the RGB components in (d)(e)(f); (k) the RGB color image after combining the RGB components in (g)(h)(i). The size of images are 330 × 330 μm (150 × 150 pixels)

Download Full Size | PDF

Since the USAF target is not a color sample, we did imaging experiment using a color sample, a stained root of the herb Saposhnikovia divaricata to demonstrate the color imaging capability of our color LIHM system. The imaging results are shown in Fig. 4

 

Fig. 4 Imaging results of the stained root of Saposhnikovia divaricataby our color LIHM under sunlight illumination. (a)The captured color hologram using the color CMOS imaging sensor; (b) the hologram under laser illumination with wavelength of 473 nm; (c) the holographic reconstruction of (b); (d)conventional microscope image observed with a 4X objective; (e)(f)(g) direct reconstruction results of the red, green and blue components of the hologram shown in (a); (h) the RGB color image after combining the RGB components in (e)(f)(g); (i)(j)(k) reconstruction results of the red, green and blue components of the hologram shown in (a)using our proposed deconvolution method;(l) the RGB color image after combining the RGB components in (i)(j)(k). The size of images are 400 × 352 μm (200 × 160 pixels)

Download Full Size | PDF

Figure 4(a) shows the captured color in-line hologram under sunlight illumination using the color CMOS imaging sensor. In comparison, Fig. 4(b) shows the in-line hologram using laser illumination with wavelength of 473 nm (MBL-III-473, Changchun New Industries Optoelectronics Technology Co.). We can see that Fig. 4(a) is much more blurry because of the insufficient temporal coherence. As baseline sample images for comparison, Figs. 4(c) and 4(d) show the holographic reconstruction result of Fig. 4(b) and a microscope image obtained with 4X objective, respectively. The direct reconstruction results of the sub-hologram components of red, blue, and green with center wavelengths of 604 nm, 537 nm, and 455 nm are shown in Figs. 4(e)(f)(g), respectively. And Fig. 4(h) is the RGB color image obtained by combing the color components in Figs. 4(e)(f)(g). Because of the broad spectral distribution and thus insufficient temporal coherence, we can see that the reconstructions look blurry compared with laser illumination result in Fig. 4(c) and the conventional microscope image in Fig. 4(d). In contrast, Figs. 4(i)(j)(k) show the reconstruction results for red, green, and blue components using our proposed deconvolution method with the abovementioned synthetic PSF. And Fig. 4(l) is the RGB color image obtained by combing the color components in Figs. 4(i)(j)(k). We can easily observe the resolution enhancement compared with the direct reconstruction. By comparing Fig. 4(d), 4(h), and 4(l), we can see that the color holographic reconstruction shows similar color distribution as the conventional microscope image. Nevertheless, the resolution and contrast of the color LIHM image are still worse than that of the microscope image in Fig. 4(d). This is mainly because of the approximation of weakly scattered amplitude object in our method.

4. Discussions and conclusions

We should notice that in our system, the short distance between the sample and the imaging sensor is critical. Because of the insufficient temporal coherence, large sample-to-sensor distance will lead to very blurry “shadow” where the high spatial frequency components of the hologram will be lost, and thus cannot be used for holographic reconstruction. This is a distinct difference compared with conventional holographic reconstructions where sufficient temporal coherence is guaranteed [8].

Another issue that affects our image reconstruction is the deconvolution process. We used a simple Wiener filter for deconvolution where the PSF is computed from the spectral responses of the RGB components. However, the spectral distribution might not be accurate because the sunlight spectrum might change in different weathers [18]. Considering this issue, better reconstructions should be possible if accurate spectral distribution is known for computing the synthetic PSF. Furthermore, it is well known that the RGB channels in the imaging sensors have crosstalk which would affect the accuracy of the spectral response used in our PSF. O. Flasseur et al. has proposed a parametric inverse problem approach to compensate for it in lensless color microscopy [19]. Z. E. Phillips et al. [20] and W. Lee et al. [21] also proposed a linear model to correct the crosstalk problem for quantitative phase microscopy with color LED array. Although we cannot directly use these approaches because of the difference in setups and illuminations, it might be possible to solve the problem and further improve our image reconstruction quality with similar methods in the future.

We also observe that one of the major limitations of our method is the assumption that the sample is amplitude object. And only in this case, the captured sub-hologram intensities can be expressed as a convolution process with the analytic expression shown in Eq. (3). Further studies are needed to evaluate whether our assumptions are too restrictive and it might be possible to loosen the approximations. Nevertheless, for samples that exhibits phase modulation, we can approximately treat it as amplitude object if the phase modulation is small. And our reconstruction can still obtain better reconstruction results, as can be seen in Fig. 4, where the sample was not a pure amplitude object.

In summary, we propose a novel color lensless in-line holographic imaging method that can be used to acquire color images of weakly scattered amplitude objects under white light illumination including sunlight. In our method, only one color in-line hologram is required which can be obtained with a color CMOS imaging sensor. The basic principle of our color LIHM is to describe the RGB components of the acquired in-line hologram as a convolution process. And the color sample image can be thus obtained by recombining the RGB components reconstructed with a deconvolution process. We demonstrated the resolution enhancement capability of the deconvolution process by performing numerical simulations and experiments with the USAF target as the sample. The color sample of stained root of Saposhnikovia divaricata was also imaged to demonstrate the color imaging capability of our system. Because of the simple setup and one-shot color imaging capability, our color LIHM method is suitable for low-cost and compact color imaging applications.