The optical memory effect describes the shift-invariance of speckle patterns when a light source illuminating a scattering surface is slightly tilted (e.g., in the case of plane wave illumination) or shifted (e.g., in the case of point source illumination) [1]. In recent times, many imaging techniques exploiting the memory effect of scattering media to image through and with visually opaque materials have been presented. These techniques apply Fienup-type algorithms [2] on the random patterns generated by illuminating objects hidden behind scattering media [3,4] or they use the knowledge of the scattering behavior of the visually opaque medium gained with known objects [5] or a reference point source [6]. The latter approach means the use of a previously recorded point spread function (PSF). In former publications, we demonstrated the capabilities of such an approach called scatter-plate microscopy (SPM) to realize high-resolution 3D microscopic imaging [7,8]. In SPM, images are retrieved by spatially cross correlating the PSF with the scattered intensity pattern generated by the illuminated object. To realize the spatial cross correlation, both patterns have to be recorded with 2D sensors (by, e.g., a CMOS or a CCD). To achieve a high resolution and good signal-to-noise ratio (SNR), a large part of the speckle patterns must be recorded. Furthermore, the sensor area should be much larger than the image of the sample. These requirements limit the achievable magnification; thus, for high-resolution microscopic imaging without too high magnification a small pixel pitch is needed. This makes the sensor bulky and expensive.

In this publication, we demonstrate that the use of a 2D sensor is actually not required to realize the concept of SPM. Instead we show that SPM can realize compressed sensing and develop a single-pixel SPM [9]. Instead of calculating a spatial cross correlation, we calculate an ensemble cross correlation, which allows us to replace the 2D sensor by a simple single photodiode. The ensemble correlation needs the recording of a single-pixel signal for many scatter-plate realizations. To realize many scatter-plates, we apply spatial light modulators (SLMs) with pseudorandom patterns. Since these patterns are known, the PSFs can be obtained by calculation and must not be measured, which speeds up the method and opens up a wide field of possible applications. Similar to the “classical” SPM, single-pixel SPM has the potential to resolve 3D [7] and spectral information [10] about the sample.

In SPM, a light-scattering layer such as the rough surface of a ground glass diffuser serves as imaging element. Positioned after a spatially incoherently illuminated sample in the beam path, the scattering element generates a random intensity pattern ${I_{{ob}}}(\textbf{r})$. An image $I(\textbf{r})$ is retrieved by calculating a spatial cross correlation between this pattern and the previously recorded intensity PSF $S(\textbf{r})$ of the scatter-plate [8]:

If we apply a scatter-plate with known scatter-matrix such as an SLM with some random but known modulation pattern $M$, it is not necessary to record the PSF; instead, it can be calculated analytically, and SPM becomes a true single shot imaging method.

To transform the method into a single-pixel imaging technique, we use the fact that due to a kind of ergodicity the spatial cross correlation is equivalent to an ensemble cross correlation obtained by averaging over single-pixel signals recorded with many scatter-plate realizations [11]. Consider the normalized ensemble cross correlation:

 

Fig. 1. Experimental setup for single-pixel SPM. The region of interest (ROI) is restricted by a pinhole. A laser beam is lead through a rotating ground glass diffuser to realize temporally coherent but spatially incoherence illumination. A lens images the spot on the rotating diffuser onto the ROI. As scatter-plate serves either a (a) transmittive SLM, or (b) an LCOS or a DMD.

Download Full Size | PDF

 

Fig. 2. Single-pixel SPM image reconstruction. For $N$ different scatter-plate realizations. The intensity signal of the single-pixel detector is recorded and stored in the intensity signal vector ${I_{{ob}}}(\textbf{R})$. For each scatter-plate realization, we further calculate the 2D-PSF in the image plane. For each pixel position $\textbf{r}$ in the image plane, we cross correlate according to Eq. (2) the vector $S(\textbf{r})$ from the resulting PSF stack with ${I_{{ob}}}(\textbf{R})$ to retrieve the image point $I(\textbf{r})$.

Download Full Size | PDF

Since images are retrieved by correlation, potentially existing random noise ($n(\textbf{R})$) in the signal measurement (${I_{{\rm measure}}}(\textbf{R}) = {I_{{ob}}}(\textbf{R}) + n(\textbf{R})$) is not dominating the image (since $\langle S(\textbf{r})n(\textbf{R})\rangle = \langle S(\textbf{R})\rangle \langle n(\textbf{R})\rangle$). The method is, therefore, applicable in low SNR measurement environments.

As in conventional SPM, the magnification of single-pixel SPM is given by $\frac{z}{{{z^\prime}}}$ (distance between scatter-plate and sensor plane over distance between scatter-plate and object plane; see Fig. 1)

To retrieve images with the described method, the PSFs have to be random speckle patterns. Any symmetries or periodicities in the PSFs may cause twin images, and intensity fluctuations larger than speckles increase the image blur. To reduce the noise, a well-resolved recording of the speckles has to be ensured. Depending on the pixel size, the number of pixels, the NA of the setup, and the distances ${z_1}$ and ${z_2}$, the condition of well-resolved speckles can be realized with both phase-only SLM and binary SLM [like, e.g., a digital micromirror device (DMD)].

We investigated the capabilities of our single-pixel SPM concept by simulations on the discrete coordinate meshes ${\textbf{r}^\prime} = (x_i^\prime ,y_i^\prime ,z_i^\prime)$ (coordinates in the object plane), ${\hat {\boldsymbol r}} = ({\hat x_i},{\hat y_i})$ (coordinates in the scatter-plate plane), and $\textbf{r} = ({x_i},{y_i},{z_i})$ (coordinates in the image plane). To simulate the point source illumination, we used a spherical wavefront to calculate the field distribution in the scatter-plate plane:

Finally, by calculating the intensity from the amplitude distribution, we obtain the intensity PSF $S({x_i},{y_i}) = {| {{E_{{im}}}({x_i},{y_i})} |^2}$.

To simulate the pattern generated by the incoherently illuminated sample, we regard each non-zero pixel of the sample as point sources. For each point source, we can determine $S({x_i},{y_i})$. The final pattern is then obtained by adding all these PSFs. Since we want to simulate single-pixel measurement, we have to calculate this pattern just at a single position $\textbf{R} = ({X_i},{Y_i})$ in the image plane. Therefore, to obtain the signal ${I_{{ob}}}(\textbf{R})$, it is sufficient to evaluate Eq. (4) just for a single position.

We simulated single-pixel SPM both with random phase-only masks and with random binary masks. Figure 3 demonstrates how the quality of the retrieved images of a monochromatic binary sample depend on the number $N$ of applied scatter-plates. Figure 3(h) shows how the SNR (${\rm SNR} = (S1 - S2)/{\sigma _b}$, with $S1$ being the mean value of the image region, S2 being the mean of the background, and ${\sigma _b}$ being the standard deviation of the background) depends on $N$. For both types of SLM, images retrieved with 10,000 scatter-plate realizations have an acceptable image quality. The result proofs that both phase-only SLM and DMD could be applied as scatter-plates to realize a single-pixel SPM.

 

Fig. 3. Simulation of single-pixel SPM. The image quality depends on the number of scatter-plates $N$. (a) As an object, we used binary letter T. We simulated single-pixel SPM with (b)–(d) phase-only SLM and (e)–(g) DMD. (h) SNR in dependency on $N$. Simulation parameters: sampling pitch 5 µm, ${\rm NA} = 0.02$, ${\lambda} = 532\;{\rm nm}$, $z = {z^\prime} = 47\;{\rm mm} $, magnification $z/{z^\prime} = 1$.

Download Full Size | PDF

Figure 4 demonstrates that also non-binary samples may be imaged. Whereas for binary sample simulation the sample point amplitude ${E_0}$ in Eq. (3) has the same value for all non-zero pixels in the sample, we simulate the image retrieval for non-binary samples with spatially modulated sample point amplitude (${E_0}(x_i^\prime ,y_i^\prime)$).

 

Fig. 4. (a) Simulation of single-pixel SPM with a non-binary object. (b) The image was retrieved with $N = {20{,}000}$ phase-only scatter-plate realizations. The other simulation parameters were the same as in Fig. 3.

Download Full Size | PDF

Single-pixel SPM also enables 3D reconstruction. The PSF of a scatter-plate depends on the axial position of the point source. We can now calculate various PSF stacks for different distances between the point source and the scatter-plate. Afterward we retrieve an image for all the point source stacks via Eq. (2). For each image pixel, we determine the point source stack, which gives the highest intensity in the image plane. This intensity should be higher than a threshold value ($0 \lt \alpha \lt 1$) to discriminate between noise and true image points. Stronger image noise requires a higher value of $\alpha$. The axial position of the point source corresponding to the PSF stack resulting in the highest image point intensity indicates now the axial $z$ coordinate of the object point. The simulation of a 3D image reconstruction of a binary sample is shown in Fig. 5.

 

Fig. 5. Simulation of depth resolved imaging with single-pixel SPM. The object (a) had two axially separated parts. One part was 47.75 mm, and the other 48.8 mm away from the scatter-plate. (b) For the depth resolved reconstruction, we simulated 51 PSF stacks with the point source to scatter-plate distances between 47 mm and 49.5 mm. The other simulation parameters were the same as in Fig. 4.

Download Full Size | PDF

With a similar approach, we can also retrieve spectral information about the sample. Instead of calculating PSF stacks for different distances of the point source, we calculate various PSF stacks for different irradiation wavelengths. Again, we have to determine for each image point with which PSF stack we retrieve the highest intensity and apply some threshold to discriminate between noise and bright image points. The wavelength of the determined PSF stack approximates the wavelength irradiated by the corresponding object point. Figure 6 shows the simulation results of spectrally resolved single-pixel SPM for a 2D sample.

 

Fig. 6. Simulation of spectrally resolved imaging with single-pixel SPM. The object (a) was split in three parts irradiating light with different wavelength. (b) For the spectrally resolved reconstruction, we simulated 21 PSF stacks with wavelengths between 400 nm and 650 nm. (c)–(e) demonstrate the reconstruction with PSFs of different wavelength. The central bar of the letter E is well reconstructed with a PSF-wavelength of 613 nm. The other simulation parameters were the same as in Fig. 4.

Download Full Size | PDF

For the experimental validation, we implemented the setup in Fig. 1(b). To realize different random scatter-plates, we used a DMD (DLP4500.45 WXGA with $912 \times 1140 \;{\rm micromirrors}$ in diamond array orientation and a micromirror pitch of 7.4 µm; see also TI, DLP4500.45 WXGA DMD datasheet.). To provide spatially incoherent monochromatic sample illumination, a frequency-doubled Nd:YAG-laser beam with a wavelength of 532 nm was let through a rotating ground glass diffuser and focused on the sample. For signal detection, we read out a single pixel of a CCD-camera from VISTEK-GMBH with a pixel size of $5.5\; {\unicode{x00B5}{\rm m}} \times 5.5\; {\unicode{x00B5}{\rm m}}$. We performed the calculations of the PSFs on a rectangular grid with 2.7 µm pixel pitch. To realize this, we first emulate the binary random mask displayed by the DMD on a diamond grid (each micromirror was treated as one pixel), and in a second step we obtained the random mask on our finer grid via nearest neighbor interpolation. Finally we applied zero padding to obtain a square grid with $3647 \times 3647 \;{\rm pixel}$_s. The binary random masks were combined with circular masks to restrict the NA (micromirrors turned into the off state outside a circular region in the center of the DMD). Figure 7(a) shows the result for monochromatic 2D imaging of the USAF target group 2 element 2. All negative image values are set to zero. Figure 7(c) shows cross line profiles from normalized images retrieved with different numbers of patterns. Setting negative image values to zero and normalizing the images improves the visibility of the signal in images retrieved with a lower number of patterns. Unfortunately a significant noise makes it impossible to use our setup for 3D or spectral data acquisition. We are confident that the problem can be overcome by realizing the setup with a phase-only SLM in an inline-configuration [see Fig. 1(a)], enabling the PSF calculation more accurately.

 

Fig. 7. (a) Experimental validation of single-pixel SPM. We imaged group 2 element 2 of the USAF chart. Binary random scatter-plates were realized with a DMD. The scattering-region was restricted by a circular aperture mask (displayed on the DMD, radius $r = 1.54\;{\rm mm} $, ${\rm NA} = 0.016$), the distance between the object and the DMD was ${z^\prime} = 85\;{\rm mm} $, and the distance between the DMD and the detector was $z = 192\;{\rm mm} $ (magnification $z/{z^\prime} = 2.26$). The image was retrieved with $N = {14{,}000}$ scatter-plate realizations. (b) Image with a lens ($f = 60\;{\rm mm} $, magnification $z/{z^\prime} = 2.26$, ${\rm NA} = 0.016$). (c) Cross line profiles along the dashed line in (a) from normalized images retrieved with 6000, 10,000, and 14,000 patterns.

Download Full Size | PDF

The single-pixel SPM we developed proves to be a powerful tool enabling lensless microscopic imaging by single-pixel signal recording. In principle, both spectral and depth information about the sample can be retrieved from the recorded intensity signal. With the application of phase-only SLM, an inline-configuration of the microscope could be realized. This would enable higher magnifications, a more precise calculation of the PSFs, and consequently the experimental validation of spectrally resolved and, respectively, depth resolved, single-pixel SLM.