1. Introduction

Light scattering prevents imaging or focusing of light beams with long penetration depths into biological tissues. Great efforts have been made to overcome this limitation through various approaches, such as improving the detection of ballistic light, developing new methods for precise wavefront control, or using diffuse optical imaging techniques [1–5].

Ballistic imaging typically provides good resolution but is severely limited in penetration depth due to the rapid attenuation of unscattered photons in scattering media. These techniques aim to discriminate scattered light based on time of arrival [6,7], direction or spatial frequency [8], polarization [9,10], or coherence [11].

Wavefront control methods offer better penetration compared to ballistic techniques [4,12], although they are computationally expensive either because they are iterative [13] or because they characterize the transmission matrix of complex media [14]. However, these methods still have limitations. One is the penetration depth inside the scattering media at which it is possible to obtain high quality images, because light eventually loses its initial coherence due to random scattering events. On the other hand, these techniques rely on specific geometric configurations that often require access to both sides of the scattering medium, limiting their practical applications.

Finally, diffuse optical imaging (DOI) techniques achieve high penetration depth at the expense of low resolution and the requirement to use cumbersome inversion methods based on physical light propagation models. The goal of this imaging modality is not to image through turbid media, but rather to characterize variations in optical absorption and scattering properties [2,15,16]. Among the traditional DOI methods using many source-detector pairs, spatial frequency domain imaging (SFDI) has recently emerged as a non-contact, broad-beam imaging technique in which the optical properties of turbid media (namely scattering and absorption) are characterized by the spatial frequency domain response of the medium [17–20].

Recently, SPI has been proposed as a useful tool for many imaging applications, including ballistic methods for imaging through scattering media [21–23]. SPI techniques based on structured illumination work by scanning the scene with a set of structured light patterns, typically by using digital micromirror devices (DMDs), while the light intensity transmitted or reflected by the object is integrated by a single-pixel or bucket detector [24,25]. Recent advances in this technique include light-field and 3D microscopy [26,27], fluorescent diffraction tomography [28], multidimensional fluorescence microscopy with data fusion [29], and ultrahigh speed single-pixel imaging [30]. SPI with structured illumination has proven to be tolerant to the presence of scattering between the object and the light sensor [21,22]. This is, in fact, one of the advantages allowing two-photon scanning microscopy to achieve deep tissue penetration, along with the use of infrared illumination [31]. However, single-pixel imaging techniques are still sensitive to scattering media located between the spatial light modulator and the object. This makes it difficult to apply these techniques to objects immersed in scattering media.

We propose a new system for imaging objects occluded by inhomogeneous scattering media, i.e., media whose optical properties vary with the spatial location, using single-pixel imaging techniques with structured illumination. It works in an adaptive way by combining DOI and SPI techniques in two stages. First, the turbid media is characterized by projecting light patterns with a programmable LED array and applying SFDI technique. Second, the LED array is programmed to project light only through areas of reduced scattering in the media, while simultaneously using a DMD to project light patterns onto the target using Hadamard basis coding functions. Using this adaptive technique, we are able to obtain images of targets through scattering media with better quality than using conventional illumination.

2. Measurement procedure

The measurement performed by a single-pixel camera can be mathematically expressed as:

Since the projection of sampling patterns is sequential in time, the acquisition time tends to be long when the number of pixels of the reconstructed image is large. This makes the acquisition time the bottleneck of this imaging technique. To overcome this problem, compressive sensing (CS) aims to reduce the acquisition time by acquiring fewer measurements than the number of pixels in the image ($M<N$) [32–34]. This signal processing technique relies on the fact that most natural images are compressible in the sense that they have a sparse representation in some basis. CS reconstructs this sparse representation from undersampled measurements. Optimally, the measurement functions should be chosen to be mostly incoherent with the sparse basis (i.e., each sampling pattern should have a non-sparse representation in the reconstruction basis). Mathematically, the reconstruction is performed by iteratively minimizing a cost function composed of a least squares data fidelity term and a $l_1$ norm regularization term that ensures the sparsity of the result. Often, however, a regularization consisting of minimizing the total variation (TV) norm of the image in the canonical or pixel domain is chosen. The optimization problem in this case is written as:

Unlike a conventional camera-based approach, the single-pixel camera is tolerant to the presence of scattering media between the object to be imaged and the detector [21]. However, the best possible projection of the light patterns through the turbid media must still be ensured. In this work, we improve the performance of a single-pixel camera in reflection geometry by projecting light patterns through the regions of the turbid media with higher transmittance of ballistic or collimated light, while avoiding areas of high absorption or scattering. This is achieved by using a programmable light source based on an array of LEDs.

To find the regions of higher ballistic transmittance of the turbid media, we use a technique based on SFDI. The standard method consists of projecting sinusoidal patterns of different spatial frequencies onto the turbid medium and capturing images of the diffuse reflected light. Due to propagation within the turbid medium, the backscattered light follows a damped sinusoidal distribution whose amplitude modulation is highly dependent on the scattering and absorption properties. Consequently, local variations in these physical parameters will result in changes in the amplitude modulation of the diffuse reflected light. It has been shown that low and high frequencies are sensitive to changes in absorption and scattering properties, respectively [17]. Therefore, to properly separate absorption and scattering parameters, at least two spatial frequencies are required, a low or zero frequency ($f_{DC}$) and a relatively high frequency ($f_{AC}$), chosen depending on the range of values of the scattering and absorption coefficients. However, since we cannot modulate negative intensity light patterns, there is always a DC offset when projecting AC frequency patterns, and therefore it is sufficient to use a single AC frequency sinusoidal pattern. In this case, the sinusoidal pattern is projected with three spatial phase offsets ($0$, $2\pi /3$, and $4\pi /3$), which allows the images of the AC and DC frequency amplitude modulations ($M_{AC}[x_i,y_j]$ and $M_{DC}[x_i,y_j]$, respectively) to be determined from the three measured reflectance images ($I_{0}(x_i,y_j),I_{2\! \pi \!/\!3}(x_i,y_j), \, \mathrm {and}\, I_{4\!\pi \!/\!3}[x_i,y_j]$) by the following three-step demodulation method:

These demodulated images are then calibrated against a reference turbid media phantom with known optical properties. This calibration corrects for the instrument response of the imaging system and allows quantitative values of the absorption and scattering coefficients to be obtained through an inversion method. This inversion method involves a light propagation model, including Monte Carlo simulations or analytical models based on diffusion theory, and least-squares fitting or look-up table interpolation. However, for a qualitative characterization, the calibration step is not necessary, since the relative degree of absorption and scattering is related to the amplitude modulations at different spatial frequencies [17]. At a low spatial frequency, absorption has a maximal effect on the reflectance and the amplitude modulation is inversely proportional to the absorption coefficient. On the other hand, at a relatively high spatial frequency, the amplitude modulation is mainly determined by scattering, resulting in a directly proportional relationship [17]. Therefore, regions with higher ballistic transmittance (which decays exponentially with the absorption and scattering coefficients) are identified with lower values in $M_{AC}[x_i,y_j]$ and higher values in $M_{DC}[x_i,y_j]$.

The imaging system presented in this paper combines both imaging modalities, DOI and SPI. It uses an array of LEDs as a programmable light source for SFDI and a DMD as a spatial light modulator for SPI. First, the turbid media is characterized by SFDI to find the areas of higher transmittance by projecting the sinusoidal patterns generated in the LED array. To apply SFDI, the LED array, the turbid media, and the camera are located at conjugated planes. Next, by using the correspondence between the LED array and the turbid media, the light source is adapted in such a way that only those individual LEDs illuminating the areas of the turbid media with higher ballistic transmittance are turned on. Therefore, the light patterns generated by the DMD propagate through the best areas in the turbid media toward the object. A single pixel detector collects the light reflected from the object. The final image is reconstructed using SPI techniques.

3. Experimental setup

Our optical system is shown schematically in Fig. 1. In this setup, we perform both imaging techniques: SFDI, which allows us to qualitatively characterize the scattering media, and SPI, to image objects hidden by these media. The light source is a $32\times 32$ LED array ($4\,\mathrm {mm}$ grid spacing, $32\times 32$ RGB LED matrix panel - $4\,\mathrm {mm}$ pitch from Adafruit), controlled by an Arduino microcontroller board (Arduino Mega 2560 Rev3). The pattern displayed by the LED array is projected onto the turbid medium by a $4f$ system consisting of lenses $\mathrm {L}_1$ and $\mathrm {L}_2$ with focal lengths of $180 \, \mathrm {mm}$ and $150 \, \mathrm {mm}$, respectively. The first step is to characterize the turbid medium using the SFDI technique, represented by the yellow beam trajectory. For this purpose, sinusoidal patterns are projected onto the turbid medium by setting all micromirrors of the DMD ($768 \times 1024$ pixels, pixel size $13.7 \, {\mathrm{\mu} \mathrm{m}}$, module V-7001 by Vialux) in the on-state, i.e. the DMD acts simply as a single flat mirror, because all micromirrors reflect the light in the direction of the optical path. The diffuse light reflected by the turbid medium is imaged by a $2048 \times 2048$ pixel CMOS camera (IDS uEye UI-3370CP-C-HQ).

 

Fig. 1. Schematic experimental setup. First (shown by the yellow path), a LED array illuminates the turbid media with a sinusoidal intensity profile. A camera images the diffuse reflected light distribution, which is used to determine the scattering and absorption regions of the medium. Second (red path), the LED array is programmed to illuminate the areas of higher transmission of the turbid media, and a set of sampling patterns is encoded on the DMD and projected onto the object. A photodiode measures a signal proportional to the reflected light. $\mathrm {L}_1$, $\mathrm {L}_2$, $\mathrm {L}_3$, and $\mathrm {L}_4$ are refractive lenses, M is a mirror, and BS is a beam splitter.

Download Full Size | PDF

Since the LED array and the turbid medium are in conjugated planes, each LED in the array illuminates a single spot on the turbid medium. Therefore, during the SPI step (shown by the red beam path), the light source is programmed to illuminate the required areas of higher transmission by turning on the required LEDs. In this case, the DMD acts as a spatial light modulator, sequentially displaying sampling patterns that scan the object. These patterns are projected onto the object by a second $4f$ optical system consisting of lenses $\mathrm {L}_2$ and $\mathrm {L}_3$ (focal length $150$ and $100 \, \mathrm {mm}$, respectively). The reflected light is redirected by lenses $\mathrm {L}_3$ and $\mathrm {L}_2$ and a beam splitter (BS) and finally collected by lens $\mathrm {L}_4$ of focal length $50 \, \mathrm {mm}$ on the photodiode (Thorlabs DET36A) which measures the intensity signal for the set of sampling patterns. Note that, as the DMD is located at the field diaphragm and the turbid media is located at the aperture diaphragm of the optical system, limiting the size of the area illuminated by the light source at the scattering media does not limit the object field of view.

The proposed system has been tested on two different inhomogeneous turbid media samples: one consisting of one or more layers of organic fiber sheets, and the other an epoxy resin slab with a non-uniform distribution of scattering $\mathrm {Ti0_2}$ nanoparticles. For the fiber sheets the value of the reduced scattering coefficient is between $156-207 ~\mathrm {cm}^{-1}$, while the absorption coefficient is in the range of $0.04-0.05 ~\mathrm {cm}^{-1}$. It is therefore a highly diffusive medium, despite its low thickness of only $0.048~\mathrm {mm}$ for a single layer and $0.095~\mathrm {mm}$ for two layers. The $2.3~\mathrm {mm}$ thick epoxy resin slab has a reduced scattering coefficient value of about $5.9~\mathrm {cm}^{-1}$ for the high scattering regions and about $3.2~\mathrm {cm}^{-1}$ for the low scattering regions. The absorption coefficient for the high and low scattering regions is $0.32~\mathrm {cm}^{-1}$ and $0.75~\mathrm {cm}^{-1}$, respectively. These parameters were determined by a double integrating sphere method and the Kubelka-Munk model [38,39]. In all the experiments presented in this paper, a spatial frequency of $0.3~\mathrm {cm}^{-1}$ is used for SFDI characterization, that has been shown to be adequate for the employed samples. Functions of the Walsh-Hadamard basis are used as sampling patterns for SPI. Images with $32\times 32$ pixels ($N=1024$) are reconstructed by the single pixel camera by projecting up to $2N$ patterns, where the factor of 2 is due to the necessity of time multiplexing each Walsh-Hadamard function into two positive-valued functions. These patterns are encoded using the maximum square aperture of the DMD, which consist of $768\times 768$ micromirrors. Binning is performed by using $24 \times 24$ micromirrors for each pixel of the Walsh-Hadamard functions. The imaged objects are two different 1951 USAF resolution test targets: one reflective and one fluorescent. Although the LED array can provide RGB color illumination, we use only one color channel at a time to avoid chromatic dispersion due to the periodic structure of the DMD. In general, we use the red color (centered at $630.8\, \mathrm {nm}$ with a bandwidth of $20\, \mathrm {nm}$) because it provides the greatest penetration depth in turbid media and is also the brightest of the three color channels. However, we also used the blue color (centered at $460\, \mathrm {nm}$ with a bandwidth of $20\, \mathrm {nm}$) to excite fluorescence.

4. Results

In the first experiment, to verify and compare our system with a conventional camera, we consider a non-homogeneous layer of fiber sheet as a turbid medium. As mentioned earlier, the imaging capabilities of a conventional camera are severely limited when a turbid medium is placed between the object and the sensor. For comparison, in Fig. 2(a) we show the images of the reflective 1951 USAF target without and with this turbid medium, taken with the same camera used for the SFDI characterization, but refocused on the object instead of the scattering medium. The conventional camera is unable to obtain an image of the object in the presence of the scattering medium. Next, we qualitatively characterized the turbid media by means of SFDI. In Figs. 2(b) and (c) we plot the qualitative spatial maps of recovered optical properties for the case of one and two overlapping layers of organic fiber sheets, respectively. In the images on the left we can recognize the higher scattering areas, while on the ones on the right we can discern areas of lower absorption. Consequently we can identify the best areas to illuminate in order to project the sampling patterns onto the object (highlighted by the red circles). In the next step we apply SPI to image the object (the positive reflective 1951 USAF target) under different illumination conditions. First, a reference image is taken without any scattering medium and with a fixed number of LEDs turned on, but in no particular order (which we will call random illumination, and which ensures the same illumination intensity without prioritizing transmission through any particular area of the medium). This reference image is shown in Fig. 2(d). Next, we locate the turbid media between the DMD and the object and repeat the imaging process with the same random illumination. These images are shown on the right of Figs. 2(e) and (f), for one and two layers of fiber sheets, respectively. Finally, we turn on the necessary LEDs adapting the programmable light source to illuminate through the areas of higher transmission of ballistic photons. The images obtained under this condition are presented on the left of Fig. 2(e) and (f) for one and two layers of fiber sheets, respectively. In both cases, with one or two layers of turbid medium, we obtain higher contrast and can see more detail when we adapt the illumination compared to random illumination. To quantify the quality improvement of the images, we calculated the contrast and the structural similarity index (SSIM). Contrast determines the image quality relative to signal-to-noise ratio while SSIM evaluates the quality with respect ground truth by comparing local patterns of pixel intensities that have been normalized for luminance and contrast [40]. To calculate these metrics we use the image in Fig. 2(d) as ground truth. The results are shown in Fig. 2(g) for the different illumination conditions and as a function of the number of layers of fiber sheets. As expected, a decrease in image quality is observed as the number of fiber sheets increases. However, the values presented confirm the improvement in image quality when using the adapted illumination for all cases.

 

Fig. 2. Experimental results obtained when layers of cotton fiber are used as a scattering medium. a) Images of the object taken with a conventional camera without the scattering medium (left) and with a single layer of scattering medium (right). The red lines delimit the area that will be imaged by SPI technique in Fig. 1. Qualitative characterization of the scattering (left) and absorption (right) properties by SFDI of b) one fiber layer and c) two fiber layers. Images of the reflective 1951 USAF target obtained with the SPI system under different illumination conditions d) without scattering medium, e) with one fiber layer, and f) with two fiber layers as scattering medium. g) Values of contrast and SSIM for the reconstructed images.

Download Full Size | PDF

To demonstrate the versatility of our setup, in the next experiment we use a different object, a fluorescent 1951 USAF resolution test target, through a single layer of fiber sheet. For this we used the blue color channel of the LED array and placed a yellow dichroic filter (Coherent 36-6674-000) between $\mathrm {L}_4$ and the BS. The results obtained are shown in Fig. 3. In Fig. 3(a) we show the reconstructed images under the different illumination conditions, where we see a higher contrast and a better recognition of the edges of the object in the image obtained with adapted illumination with respect to that obtained with random illumination. In addition, in Fig. 3(b) we plot the respective values of contrast and SSIM to estimate the improvement considering the adapted and random illumination.

 

Fig. 3. Experimental results obtained by our imaging system in Fig. 1 when imaging a fluorescent USAF target through one layer of fiber sheet. a) Reconstructed images with and without turbid medium and under different illumination conditions. b) Plot of contrast and SSIM for the images in (a) showing better values for adaptive illumination.

Download Full Size | PDF

To verify the system with another type of scattering medium and to compare it with a conventional camera, we replaced the fiber sheets with an epoxy resin slab with a non-homogeneous distribution of scattering nanoparticles and repeated the experimental process. In order to compare, in Fig. 4(a) we show the images of the reflective 1951 USAF target without and with this turbid medium. From this result, it is clear that the conventional camera cannot image the object in the presence of the epoxy resin. Figure 4(b) displays the qualitative estimation of the optical properties of the slab, from which we can infer the areas with higher ballistic transmission. In Fig. 4(c) we show the reconstructed images of the object under different conditions, without scattering media, with adapted illumination and with random illumination, from left to right. The field of view of the single pixel camera corresponds to the area marked with red lines in 4(a). It is clear that we obtain a better quality image when we adapt the illumination to the areas with less scattering and absorption in order to image with the SPI technique. To quantify how much the image improves under adapted illumination, we show the image quality metrics in Fig. 4(d). The results indicate that adapted illumination yields higher contrast and SSIM values compared to random illumination.

 

Fig. 4. Experimental results obtained when an epoxy resin slab with inhomogeneous distribution of $\mathrm {Ti0_2}$ nanoparticles is used as a turbid medium. a) Images taken with the conventional camera without the scattering medium (left) and with the scattering medium (right). The area of the proposed SPI system is marked by the red lines. b) Spatial distribution of the optical properties of the epoxy resin slab qualitatively characterized with SFDI by using the first stage of our optical system in Fig. 1(c) Reconstructed images with the SPI technique by using the second stage of the optical system without turbid medium (left) and with turbid medium with adapted (middle) and random (right) illumination. d) Comparison of the contrast and SSIM quality metrics of the images in (c).

Download Full Size | PDF

In a final set of experiments, and to show that our idea is compatible with additional computational imaging techniques, we use CS to reduce the number of patterns to sample the scene and thus the acquisition time. Due to the relatively small number of pixels, the maximum acceptable compression ratio was $50{\%}$. The reconstructed images of the reflective USAF target are shown in Fig. 5(a), without turbid medium, and through the epoxy slab with adapted and random illumination. The image quality metrics are shown in Fig. 5(b). The most remarkable fact is that in the case of random illumination, the algorithm is not able to reconstruct the details of the image due to the lack of information caused by the low SNR and the undersampled measurement. This result further emphasizes the importance of using adapted illumination when using CS.

 

Fig. 5. Experimental results provided by our imaging system for a reflective 1951 USAF target, with similar illumination conditions as those used in Fig. 4, but reducing the number of measurements using CS with a compression ratio of $50{\%}$. a) Images obtained without turbid medium and through the epoxy resin slab with adapted and random illumination. b) Plots of contrast and SSIM for the different conditions.

Download Full Size | PDF

5. Conclusions

We have presented an imaging system based on SPI techniques for imaging through non-homogeneous turbid media. The proposed system combines the tolerance of SPI to the presence of a scattering medium between the object and the single pixel detector with a prior characterization of the scattering medium with SFDI to project the structured light patterns through the areas of higher ballistic transmittance. A LED array conjugated to the plane of the scattering medium is used for selective illumination. As this system is designed to find the areas of higher ballistic transmittance in non-homogeneous samples, improving in this way the contrast of the scanning light patterns projected onto the object and, accordingly, the quality of the final image, the idea may be not suitable for uniform ground-glass or tissue mimicking phantoms.

The imaging system was tested with different turbid media, ranging from layers of fiber sheets to an epoxy resin slab with a non-homogeneous distribution of scattering $\mathrm {Ti0_2}$ nanoparticles. We also demonstrated the versatility of the illumination by performing fluorescence imaging using the different color channels of the RGB LED array, and applied CS to reduce the number of measurements by up to $50{\%}$.

In all our results, it is apparent that we get an image with better quality when we adapt the illumination to the areas with less scattering and absorption by using the single-pixel imaging technique. Note that the quality of the image obtained with the single-pixel camera is always better than that obtained by the conventional pixelated camera. This is because the single-pixel camera filters out part of the light scattered by the turbid media when light travels from the light source to the object, due to the geometry of the detection scheme, and it is tolerant to the scattering when light travels back from the object to the detector, in contrast with the conventional camera.

The improvement of the obtained images by adjusting the illumination to the areas of higher ballistic transmission was demonstrated by calculating quantitative quality metrics such as contrast and SSIM. The results show an improvement in both metrics for each of the turbid media used. Furthermore, in the case of the application of CS, the approach of adapting the illumination is crucial to recover any information at all.

Future work could be devoted to speeding up the measurement process by multiplexing over the different color channels of the RGB LED array, but this would ultimately require correcting for the chromatic dispersion of the DMD. As well as applying CS, it would be possible to use deep learning approaches to further reduce the amount of measurements and improve the quality of the reconstructed images by reducing the noise. Wavefront control techniques could also be used to image through the turbid medium, which would require the use of a more sophisticated light source, such as a coherent source and a SLM. From an application perspective, these results open new possibilities for ocular fundus imaging through non-homogeneous cataracts. This is of uttermost importance for the diagnosis of various ocular diseases, which are often hindered by cataract-related light scattering.