Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Dual-task convolutional neural network based on the combination of the U-Net and a diffraction propagation model for phase hologram design with suppressed speckle noise

Open Access Open Access

Abstract

In this paper, a dual-task convolutional neural network based on the combination of the U-Net and a diffraction propagation model is proposed for the design of phase holograms to suppress speckle noise of the reconstructed images. By introducing a Fresnel transmission layer, based on angular spectrum diffraction theory, as the diffraction propagation model and incorporating it into U-Net as the output layer, the proposed neural network model can describe the actual physical process of holographic imaging, and the distributions of both the light amplitude and phase can be generated. Afterwards, by respectively using the Pearson correlation coefficient (PCC) as the loss function to modulate the distribution of the amplitude, and a proposed target-weighted standard deviation (TWSD) as the loss function to limit the randomness and arbitrariness of the reconstructed phase distribution, the dual tasks of the amplitude reconstruction and phase smoothing are jointly solved, and thus the phase hologram that can produce high quality image without speckle is obtained. Both simulations and optical experiments are carried out to confirm the feasibility and effectiveness of the proposed method. Furthermore, the depth of field (DOF) of the image using the proposed method is much larger than that of using the traditional Gerchberg-Saxton (GS) algorithm due to the smoothness of the reconstructed phase distribution, which is also verified in the experiments. This study provides a new phase hologram design approach and shows the potential of neural networks in the field of the holographic imaging and more.

© 2022 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

By modulating the phase component of the light utilizing the phase hologram, a holographic image system can achieve any desired intensity distribution with simple optical system. This technique is applied in many fields, such as 3D display [1,2], optical trapping [3,4], laser beam shaping [5,6], and microscopy imaging [7,8], especially with the development of the microfabrication and the spatial light modulator (SLM).

However, the coherence of the illumination light and the random phase distribution bring in considerable speckle noise [9], which degrades the reconstructed image quality greatly. Many methods have been studied to suppress the speckle noise. Some researchers have focused on decreasing the coherence of the light in temporal or spatial domain by using light-emitting diode (LED) or using time-multiplexing technique [1016]. However, the use of LED increases the complexity of the holographic image system, and the time-multiplexing methods cost a lot of time in recording multiple images [17]. Some other methods centered on constraining the phase distribution of the reconstructed image by using random-free phase such as constant, quadratic, conical phase etc. For example, C. Chang et al. presented an iteration algorithm named double-constraint Gerchberg-Saxton (DCGS) in which the phase information was constrained by a constant phase in each iteration to reduce the phase fluctuation between adjacent pixels [18]. H. Pang and L.Chen et al eliminated the speckle noise by designing an object-dependent quadratic phase distribution and imposing it on the desired image to reduce the randomness of the phase distribution [1921]. Instead of using the random phase, T. Shimobaba et al introduced a method called random phase-free computer-generated hologram by multiplying the object light with the virtual convergence light [22]. P. Tsang et al. proposed to create a phase mask with a periodic phase pattern to add to the source image and converted it into a hologram to suppress the speckle noise [23]. M.Cruz developed this method by using a random-repeated and displaced phase (RRDP) pattern to reduce the symmetry of the phase and improve the quality of the image [24]. However, these methods either need parameters adjustments for different target images, or the calculation processes are relatively complicated, which means excess time investment and the need for experienced designers.

In recent years, with the development and progress of computational science and hardware, artificial intelligence and especially deep learning have been widely used in many fields of scientific research and engineering technology [2529]. Unlike traditional machine learning, deep learning uses multi-layers neural networks to analyze data, extract features, and make decisions, so that it can use existing data to learn and solve complex problems. In the field of speckle noise reduction of the holographic images, deep learning has also gained some applications and many meaningful works have been published. For instance, W. Jeon et al introduced a multi-scale convolutional neural network (CNN) to reduce the speckle noise for a digital holographic imaging system. The network had multi-sized kernels and was trained with a large image dataset consisting of a noisy image and clean image pairs to capture and separate speckle noise component from reconstructed images [30]. D. Yin et al combined the classical GS algorithm and CNN to improve the image quality for coherent imaging. By using GS algorithm with different initial phases, two different phase holograms of the same object were generated, which were then loaded onto SLM to obtain a pair of reconstructed images. The CNN was trained by calculating the maximum likelihood estimate of pairs of images and the denoising model was finally obtained to reduce the speckle noise [17]. D. Y. Youl et al proposed a denoising convolutional neural network (DnCNN) to suppresses the speckle noise in the light field data which was transformed from the existing hologram. By using the processed light field data and the non-hogel based computer generated hologram (CGH) technique which is free from the tradeoff between the angular resolution and the spatial resolution, the new holograms could be synthesized to realized speckle-free reconstructions [31]. However, the above-mentioned works mainly focus on the image denoising by using the neural network trained on a large amount of dataset and are essentially image processing. When it comes to generating a phase hologram that can suppress the speckle noise of the reconstructed image by using the neural network directly, these methods are not applicable, and related work has not been reported.

In this paper, a phase hologram generation method based on a dual-task convolutional neural network for the speckle noise suppression of the reconstructed images is proposed taking full advantage of deep learning in solving complex problems. Different from the traditional convolutional neural network, a novel neural network model and training strategy are designed here to solve the practical problem. By incorporating a Fresnel transmission layer, based on angular spectrum diffraction theory, into U-Net, the proposed neural network model can describe the actual physical process of holographic imaging. A proposed loss function combining the Pearson correlation coefficient (PCC) loss function with the target-weighted standard deviation (TWSD) loss function is used to train the network to modulate the distribution of the amplitude and limit the randomness of the reconstructed phase distribution respectively. The phase hologram with suppressed speckle noise can be obtained after the training. Compared with the traditional GS algorithm, DCGS algorithm and other design methods, the proposed method is much simpler and more effective. What is more, our method does not require manual adjustment of the design parameters for different target images, enhancing its general applicability. The feasibility and effectiveness are verified by simulation and experimental results, which will be presented in detail in the following sections, as well as the construction and training strategy of the proposed neural network.

2. Principle and method

The schematic diagram of the proposed method is shown in Fig. 1. It can be seen from the figure that our proposed convolutional neural network contains U-Net and a Fresnel transmission layer. Two different loss functions are introduced to solve the dual-task problem. The details are explained below.

 figure: Fig. 1.

Fig. 1. The brief architecture of the neural network structure. The blocks mean the data matrixes. The red hollow arrow means the linear normalization operation. The red arrows mean the convolution, Leak ReLu activation function and a batch normalization (BN) operation. The blue arrows mean the transposed convolution, Leak ReLu activation function and BN operation. The red dash arrows mean the concatenate operation. The green arrow means the transposed convolution, tanh activation function and linear normalization operation. The black lines mean the data transmission and the black dash arrow means the data feedback to the proposed neural network.

Download Full Size | PDF

2.1. Building of the U-Net

The U-Net consists of several down-sampling blocks and the same number of up-sampling blocks. The number of the blocks G is dependent on the input image size. For convenience of discussion, we assume the image has the same size N in both directions and satisfies the following Eq. (1).

$$N = {2^G}\; ,\; $$

Each down-sampling block is used to emphasize features of the input data through convolution kernels and is composed of a convolution layer, an activation function, and a batch normalization (BN) operation, marked as the red arrows shown in Fig. 1. In this article, the kernel size in each convolution layer is set to be 4×4, and the stride is set to be 2 with zero padding so that the output matrix of each block is exactly half the size of its input matrix. Combining the Eq. (1) we can get that the size of the feature map is 1×1 in the bottleneck of the U-Net. Leaky ReLu is used as the activation function [32], and the BN operation is used in an attempt to stabilize and accelerate the training in deep neural network.

The up-sampling block is similar with the down-sampling block, except that each contains a transposed convolution layer instead of the convolution layer, marked as the blue arrows shown in Fig. 1. Meanwhile, the transposed convolution layer is identical to the convolution layer in terms of the kernel size, the stride and padding, making the output matrix of each up-sampling block exactly twice the size of its input matrix. What is more, the skip connections, between the down-sampling and it’s corresponding up-sampling blocks shown in Fig. 1, are used to avoid the decrease of the feature details. We use $f(\mu )$. shown in the following Eq. (2) as the final activation function to limit the output value of the U-Net $\varphi ({x,y} )$ between 0 and 1. The expression of the tanh function is also shown in the Eq. (2).

$$\left\{ {\begin{array}{{c}} {f(\mu )= \frac{{tanh (\mu )+ 1}}{2}}\\ {tanh (u )= \frac{{{e^\mu } - {e^{ - \mu }}}}{{{e^\mu } + {e^{ - \mu }}}}\; \; } \end{array}} \right.\; ,$$

2.2. Fresnel transmission layer

To describe the actual physical process of holographic imaging, a Fresnel transmission layer is proposed as the diffraction propagation model to overlay the U-Net to calculate the complex amplitude of the light field based on the angular spectrum diffraction theory. It is the output layer of the ovell neural network and can be expressed as the following Eq. (3).

$$U({x,y,z} )= {\mathrm{{\cal F}}^{ - 1}}\left\{ {\mathrm{{\cal F}}\{{\textrm{exp} [{j\pi \varphi ({x,y} )} ]} \}\textrm{exp} \left[ {j2\pi z\sqrt {\frac{1}{{{\lambda^2}}} - {f_x}^2 - {f_y}^2} \; } \right]} \right\}\; ,$$
Where $\mathrm{{\cal F}}$ and ${\mathrm{{\cal F}}^{ - 1}}$. represent the Fourier transform and inverse Fourier transform respectively, U is the complex amplitude of the light field, $\varphi ({x,y} )$ denotes the output of the U-Net with the size of N×N, ${f_x}$. and ${f_y}$. are spatial frequencies, λ is the wavelength, and z is the transmission distance. Considering the image size, the sampling number is also N×N, and ${f_x}$ a ${f_y}$. can be expressed as
$${f_x} = {f_y} = \frac{n}{L}\; ,$$
where $n ={-} \frac{N}{2},\; - \frac{N}{2} + 1, \ldots ,\frac{N}{2} - 1$. L is the actual width of the hologram, namely the product of the sampling numbed the size of each hologram pixel. It can be seen from the equation that the size of thevall network output U is also N×N.

2.3. Design of loss function

In order to solve the dual tasks of the amplitude reconstruction and phase smoothing, we first split the complex amplitude data into the amplitude component ${A_{m,n}}$. and the phase component ${P_{m,n}}$ using Euler formula.

$$\left\{ {\begin{array}{{c}} {{U_{m,n}} = {a_{m,n}} + {b_{m,n}}i}\\ {{A_{m,n}} = \sqrt {a_{m,n}^2 + b_{m,n}^2} }\\ {{P_{m,n}} = {{\tan }^{ - 1}}\left( {\frac{{{b_{m,n}}}}{{{a_{m,n}}}}} \right)} \end{array}\; ,\; \; \; } \right.$$

On one hand, we use the Peson correlation coefficient (PCC) as the loss function for the intensity reconstruction task by using the amplitude component ${A_{m,n}}$. . The formula of PCC is shown as the following Eq. (6).

$${L_a} = \frac{{\mathop \sum \nolimits_{m,n = 1}^N ({{A_{m,n}} - \bar{A}} )({{y_{m,n}} - \bar{y}} )}}{{\sqrt {\mathop \sum \nolimits_{m,n = 1}^N {{({{A_{m,n}} - \bar{A}} )}^2}} \sqrt {\mathop \sum \nolimits_{m,n = 1}^N {{({{y_{m,n}} - \bar{y}} )}^2}} }}\; ,\; $$
where ${y_{m,n}}$. denotes the intensity distribution matrix of the desire image. $\bar{A}$ and $\bar{y}$. denote the average values of ${A_{m,n}}$. and ${y_{m,n}}$. respectively.

On the other hand, a loss fution called target-weighted standard deviation (TWSD) is proposed to limit the randomness and arbitrariness of the reconstructed phase distribution. The calculation method of TWSD is explained as follows.

Firstly, we calculate the phase differences between adjacent rows using the following Eq. (7).

$$\Delta P_{m,n}^r = \left\{ {\begin{array}{{c}} {{P_{m + 1,n}} - {P_{m,n}}\; ,\; ({m = 1,2, \ldots ,N - 1\; ;\; n = 1,2, \ldots ,N} )}\\ {0\; ,\; \; ({m = N;\; n = 1,2, \ldots ,N} )} \end{array}} \right.\; ,$$
where P denotes the reconstructhase distribution calculated by (5). We assign a value of 0 to $\Delta P_{N,:}^r$. to ensure $\Delta {P^r}$. and P. have the same size. Taking the intensity of the target image as the weight, we deal with $\Delta {P^r}$. by the following Eq. (8) and get $\Delta {P^{r\_w}}$..
$$\left\{ {\begin{array}{{cc}} {\Delta P_{m,n}^{r\_w} = \bar{B}\; ,\; \; \; \; \; \; \; \; \; \; ({where\; \; \; \; \; {y_{m,n}} = 0} )}\\ {\Delta P_{m,n}^{r\_w} = \Delta P_{m,n}^r\; \; ,\; \; \; ({where\; \; \; \; \; {y_{m,n}} \ne 0} )} \end{array}\; ,\; } \right.$$

Here, $\bar{B}$ denotes the average value of the data of $\Delta {P^r}$. at the pixels where y≠0. Unlike $\Delta {P^r}$, the value of $\Delta {P^{r\_w}}$. is replaced by $\bar{B}$. at the pixels where y = 0, and thus the average value of the ix an be calculated by the following Eq. (9).

$$\left\{ {\begin{array}{{c}} {\overline {\Delta {P^{r\_w}}} = \frac{{\alpha \bar{B} + \beta \bar{B}}}{{N \times N}}\; }\\ {\alpha + \beta = {N^2}} \end{array}} \right.\; ,$$
where $\overline {\Delta {P^{r\_w}}} $. denotes tge . value of the matrix $\Delta {P^{r\_w}}$. . $\alpha $. and $\beta $ denote the total number of theixels whe = 0 and y≠0, respectively. Obviously, we can obtain:
$$\overline {\Delta {P^{r\_w}}} = \overline {B}, $$

This means that the average value of $\Delta {P^{r\_w}}\; $. is determined by the average value of the data of $\Delta {P^r}$. at the pixels where y≠0. In other word, the phase differences in the region where the target intensity is 0, i.e., the black area of the image, are not fed back to the neural network. This is easy to understand that the speckle noise caused by the randomness of phase has little effect when the intensity is 0.

Then we calculated the standard deviation of $\Delta {P^{r\_w}}$. with the following expression:

$$\Delta P_{std}^{r\_w} = \frac{1}{N}\sqrt {\frac{{\mathop \sum \nolimits_{m,n}^N {{({\Delta P_{m,n}^{r\_w} - \overline {\Delta {P^{r\_w}}} } )}^2}}}{{N \times N}}} \; , $$

Moreover, the phase differenc matrix between adjacent columns $\Delta {P^c}$. can be calculated and transformed to $\Delta {P^{c\_w}}$. the same way, and the standard deviations $\Delta P_{std}^{c\_w}$ is obtained. Finally, the TWSD is calculated using the following expression for the arithmetic mean:

$${L_p} = \frac{{\Delta P_{std}^{r\_w} + \Delta P_{std}^{c\_w}}}{2}\; ,$$

Since we have dual tasks to be solved, a parameter $\omega $. is introduced to balance the weight of each task.

$$T = \omega {T_a} + ({1 - \omega } ){T_p}\; ,\; $$
where T. denotes the entireoss function of the task. ${T_a}$. and ${T_p}$. denote the amplitude reconstruction task and phase smoothing task respectively. If $\omega = 1$., the network is focusing on the amptude, and the result is similarly to that of GS with totally random phase distribution. If $\omega = 0$, the network is focusing on the phase distribution, leading to a large deviation between the reconstructed image and the desired image. Here we take $\omega = 0.5$, which has aatisfactory balance and an optimal result.

One more thing to note here is that we divide the entire image into a concern domain and unconcern domain to achie better convergence. The concern domain is in the center of the image with the siz ich is slightly smaller than the image size N. ${T_a}$ and ${T_p}$ are the values of the ${L_a}$ and ${L_p}$ in the concern domain respectively, meaning that the amplitude reconstructn and phase smoothing tasks are just solved in the concern domain.

2.4. Training of the network

Finally, the target image is fed to the network we build, and the Adam optimizer is adopted to optimize the weights and biases. When the training process is completed, the U-Net output $\varphi ({x,y} )$ is obtained as the phase hologram of the desired image.

3. Simulation and experiment

3.1. Simulation

To demonstrate the performance of the proposed method, three different images as shown in Fig. 2 are employed to design the phase holograms.

 figure: Fig. 2.

Fig. 2. The target images, (a) “Square”, (b) “Football”, and (c) “Cartoon Girl”. The concern domains are marked with the yellow dashed boxes.

Download Full Size | PDF

Each of the images comprises 1024×1024 pixels, namely N = 1024. We set Nc = 800 here. The wavelength λ is 650 nm and the sampling interval is 8×8 µm, considering the models of the laser and SLM we actually use. The calculation width of diffraction field L is 8µm×1024 = 8.192 mm and the transmission distance z is set to be 120 mm. The number of down-sampling blocks is $G = {\log _2}1024 = 10$, and the number of up-sampling blocks is also G ensuring the output of the U-Net has the same size of the input image.

We build the network using the formulas and parameters mentioned in the previous section and train the network to get the phase hologram with suppressed speckle noise. The intensity and phase distribution of the reconstructed image is calculated by the angular spectrum diffraction theory. The proposed neural network is implemented in Python 3.8.11 based on the PyTorch version 1.9.0 platform on a computer with an Intel Xeon Gold 6226R processor, 256 GB main memory, and an NVIDIA GeForce RTX 3090 GPU with CUDA version 11.2. The Adam optimizer with a learning rate of 0.0001 is adopted to optimize the weights and biases. It takes about 20 minutes with 4000 epochs. For comparison, the traditional GS algorithm and DCGS algorithm are also used to design the phase hologram. The results of the “Square” image are shown in Fig. 3.

 figure: Fig. 3.

Fig. 3. The simulation results of the image “Square”; (a)-(c) The phase hologram, the reconstructed image, and the reconstructed phase distribution by GS algorithm (First row), DCGS algorithm (Second row), and our proposed method (Third row); (d) The cross-sectional diagram of the phase distribution along the dashed line in (c); (e) The cross-sectional diagram of the normalized intensity of the reconstructed images along the dashed lines in (b); (f) The graph of the loss function during training.

Download Full Size | PDF

It can be seen from the first row of Fig. 3(c) and Fig. 3(d) that the phase distribution of the reconstructed image using GS algorithm is quite random, and its intensity distribution suffers from speckle noise due to the destructive interference between the adjacent pixels, as shown in the first row of Fig. 3(b) and the red line in Fig. 3(e). This is as we expected. The DCGS algorithm can limit the randomness of the phase distribution and restrict it to a constant value, as shown in the second row of Fig. 3(c) and Fig. 3(d). Although the speckle noise is suppressed, the quality of the reconstructed image deteriorates due to some artifacts as shown in the second row of Fig. 3(b) and the blue line in Fig. 3(e). The main reason for this phenomenon is that the fixed value of the phase limits the reconstruction of the intensity. When it comes to the proposed method, since the dual tasks of the image reconstruction and phase smoothing have been solved, the reconstructed phase distribution is smoothed in the regions where the intensity of the image is not zero, and the reconstructed image has high quality with less speckle noise as shown in the third row of the Fig. 3(b) and the green line in Fig. 3(e). The neural network converges after 4000 epochs as shown in Fig. 3(f).

The simulation results of the “Football” image are shown in Fig. ( 4), which are similar to those of the “Square” image.

Unlike the “Square” image and “Football” image which are binary images, the “Cartoon Girl” image is a grayscale image, which contains much more details and can be used to further verify the effectiveness of our method. The simulation results are shown in Fig. 5.

 figure: Fig. 4.

Fig. 4. The simulation results of the image “Football”; (a)-(c) The phase hologram, the reconstructed image, and the reconstructed phase distribution by GS algorithm (First row), DCGS algorithm (Second row), and our proposed method (Third row); (d) The cross-sectional diagram of the normalized intensity of the reconstructed images along the dashed lines in (b); (e) The graph of the loss function during training.

Download Full Size | PDF

 figure: Fig. 5.

Fig. 5. The simulation results of the image “Cartoon Girl” ; (a)-(c) The phase hologram, the reconstructed image, and the reconstructed phase distribution by GS algorithm (First row), DCGS algorithm (Second row), and our proposed method (Third row); (d) The cross-sectional diagram of the phase distribution along the dashed line in (c); (e) The cross-sectional diagram of the normalized intensity of the reconstructed images along the dashed lines in (b); (f) The graph of the loss function during training.

Download Full Size | PDF

The first row of Fig. 5 shows the results of the GS algorithm, which contains a reconstructed image with speckle noise and a completely random reconstructed phase distribution. The second row of Fig. 5 shows that the DCGS algorithm achieves the suppression of the speckle noise by constraining the phase to a constant. In the region where the image intensity is not zero, the maximum phase difference is only 0.14π. However, the near-constant phase distribution limits the intensity reconstruction ability of the hologram, and the quality of the reconstructed image is affected by artifacts, just the same as the result of the “Square” and “Football” image. Compared to these two algorithms, the proposed method shows great advantages in image reconstruction and phase smoothing as shown in the third row of Fig. 5. Although the overall phase distribution still fluctuates considerably, i.e., nearly π, in the region where the image intensity is not zero, the maximum phase difference between the adjacent pixels is only about 0.008π through calculation. This ensures that both the reconstructed image and the noise suppression can achieve ideal results.

To make a quantitative comparison among the three methods, the peak signal-to-noise (PSNR) between the reconstructed image and the target image is calculated and given in Table 1. The higher value meaning the better quality of the reconstruction. Evidently, the proposed method has the best scores of PSNR for all the three different images as shown in Table 1. It is also worth noting that the PSNR value of our proposed method is still very high for grayscale images, which demonstrates the potential for solve complex problems.

Tables Icon

Table 1. The PSNR of the reconstructed images with three different methods

3.2. Experiment and discussion

To experimentally verify the proposed method, we implement an optical system based on the phase only SLM Holoeye Pluto, as shown in Fig. 6(a). The pixel number and pixel size of the SLM are 1920×1080 and 8×8 µm, respectively. A laser with the wavelength 650 nm is used as the light source and is expanded before passing through a linear polarizer and a beam splitter (BS). The main function of the polarizer is to ensure that the beam polarization state is aligned with that of the SLM. The hologram is loaded on the SLM through software, and the HR16000CTLGEC camera with resolution of 4896×3248 and pixel size of 7.4×7.4 µm is used to capture the reconstructed image at the imaging plane. The experimental results are shown in Fig. 6(b)-(d).

 figure: Fig. 6.

Fig. 6. (a) The schematic diagram of the optical setup for the experiment; and the experimental results of the reconstructed image by GS algorithm (b); DCGS algorithm (c); and our proposed method (d).

Download Full Size | PDF

As can be seen from Fig. 6(b) the experimental reconstructed images by the GS algorithm are degraded due to the severe speckles. When the target image is the “Cartoon Girl” which contains grayscale information, the reconstructed image lost most of the details and features. As shown in the close-up view of the “Cartoon Girl”, the details on the girl’s eyes, mouth, and the skirt are indistinguishable. The results of the DCGS algorithm are shown in Fig. 6(c). Although the speckle noise is suppressed to a certain extent, and the image details are richer than that of the GS algorithm, the quality of the image edges is quite poor. This is mainly due to the fact that the DCGS algorithm limits the phase of the region where the image intensity is not zero to a constant value, making this region similar to an aperture, and the diffraction effect of the edge directly leads to the degradation of the edge quality. As a comparison, the reconstructed images of the phase hologram designed by our proposed method are shown in Fig. 6(d). It can be seen that the speckle noise is suppressed. The images have high visual quality and contains most of the details. As shown in the close-up view of the “Cartoon Girl”, the light and shadows information on the girl’s eyes and the patterns on the skirt could be easily distinguished. We also calculate the PSNR between the actual reconstructed image and the original image of the “Cartoon Girl” image. The values are 12.79 dB, 12.49 dB and 14.73 dB for the GS algorithm, DCGS algorithm and the proposed method respectively. Obviously, the proposed method has a higher PSNR value than the other two methods, meaning the high-quality image reconstruction of our proposed method. It should be noted that all the actual values have deteriorated compared to the simulated results shown in Table 1. This mainly due to the background noise of the image captured by the CCD, which is caused by the non-100% filling factor of the SLM, the ambient light and the current noise of the CCD. The “Square” and “Football” images have the similar results with 8.56 dB and 11.1 dB for GS algorithm, 8.51 dB and 9.70 dB for DCGS algorithm, 8.77 dB and 12.50 dB for our proposed respectively. These experimental results show the effectiveness of our proposed method in speckle noise suppression and high-quality image reconstruction.

In addition of the image quality, the depth of field (DOF) of the reconstructed image is also analyzed. The phase hologram is designed at z = 120 mm and loaded on the SLM. The CCD moves within a distance of 110 mm to 130 mm from the SLM and captures the reconstructed images every 5 mm. Meanwhile, the PCC between the reconstructed “Cartoon Girl” and the target image are calculated. The results are shown in Fig. 7.

 figure: Fig. 7.

Fig. 7. (a)-(e) The experimental results of the reconstructed images using GS algorithm (First row) and our proposed method (Second row) at z = 110 mm, 115 mm, 120 mm, 125 mm, 130 mm, respectively. The calculated PCC values are also shown in the lower left corner of each image.

Download Full Size | PDF

It can be seen from the first row of the Fig. 7 that the deterioration of the reconstructed images by the GS algorithm is serious when the imaging distance deviates from the design value z =120 mm by ±5 mm. The PCC values decrease from 0.72 to 0.61 and 0.62. When the distance is 110 mm or 130 mm, the details of the reconstructed image are totally lost, and the PCC values are 0.53 and 0.55 respectively. The main reason is that the random phase distribution of the reconstructed image will change drastically during the transmission process, resulting in the deterioration of the intensity even if the position of the imaging plane has a small change. As a comparison, the results using the proposed method are shown in the second row of the Fig. 7. The reconstructed images remain high quality and the deterioration is quite slight when the actual imaging distance changes from 110 mm to 130 mm. The PCC values decrease from 0.80 to 0.71 and 0.76 respectively when the actual imaging distance deviates from the design imaging distance by ±5 mm. Even when z = 110 mm and 130 mm, the PCC values are 0.67 and 0.72 respectively, much higher than that of the GS algorithm. This is mainly due to the fact that the smoothness of the reconstructed phase distribution directly contributes to the stability of the intensity during the light transmission process, making the DOF of the proposed method much larger than that of the traditional GS algorithm. This is a great advantage for the practical application of holographic devices.

4. Conclusion

In this paper, a dual-task convolutional neural network is built for the phase hologram design to eliminate speckle noise of the reconstructed image. The novel neural network model and training strategy are designed by introducing a Fresnel transmission layer and two different loss functions named PCC and TWSD, which are used to modulate the distribution of the amplitude and limit the randomness of the reconstructed phase distribution respectively. The parameters of the proposed neural network as well as the calculation methods of the loss functions are described in detail. In addition, three different target images including two binary images and a grayscale image are used for the simulation, and the results show the effectiveness of the proposed method in the phase smoothing and amplitude reconstruction. The PSNR of the reconstructed image is much better than that using the traditional GS algorithm and DCGS algorithm. Optical experiments are carried out and the results are consistent with the simulation results. Especially for grayscale image, the reconstructed image of the proposed method contains more details than that of the other two methods. At the same time, it is experimentally verified that the proposed method also has the advantage in terms of the DOF. Compared with other methods, this method is simple, efficient, and effective, and provides a new way for phase hologram design and can be extended to the case of 3D hologram with suppressed speckle noise and high-quality reconstructed image.

Funding

Chongqing Science and Technology Commission (cstc2019jscx-mbdxX0019); National Natural Science Foundation of China (6217032539); National Key Research and Development Program of China (2017YFB1002902).

Acknowledgments

The authors thank Professors PengHua Li at Chongqing University of Posts and Associate Researcher Linlong Tang for providing valuable advice.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. J. H. Park, “Recent progress in computer-generated holography for three-dimensional scenes,” J. Inf. Disp. 18(1), 1–12 (2017). [CrossRef]  

2. S. Lin, D. Wang, Q. Wang, and E. Kim, “Full-color holographic 3D display system using off-axis color-multiplexed- hologram on single SLM,” Opt. Lasers Eng. 126, 105895 (2020). [CrossRef]  

3. H. Chen, Y. Guo, Z. Chen, J. Hao, J. Xu, H.-T. Wang, and J. Ding, “Holographic optical tweezers obtained by using the three- dimensional Gerchberg-Saxton algorithm,” J. Opt. 15(3), 035401 (2013). [CrossRef]  

4. Y. Cai, S. Yan, Z. Wang, R. Li, Y. Liang, Y. Zhou, X. Li, X. Yu, M. Lei, and B. Yao, “Rapid tilted-plane Gerchberg-Saxton algorithm for holographic optical tweezers,” Opt. Express 28(9), 12729–12739 (2020). [CrossRef]  

5. M. Ploschner and T. Cizmar, “Compact multimode fiber beam-shaping system based on GPU accelerated digital holography,” Opt. Lett. 40(2), 197–200 (2015). [CrossRef]  

6. L. Ackermann, C. Roider, and M. Schmidt, “Uniform and efficient beam shaping for high-energy lasers,” Opt. Express 29(12), 17997–18009 (2021). [CrossRef]  

7. X. Lai, H. Tu, Y. Lin, and C. Cheng, “Coded aperture structured illumination digital holographic microscopy for superresolution imaging,” Opt. Lett. 43(5), 1143–1146 (2018). [CrossRef]  

8. B. Kemper and G. V. Bally, “Digital holographic microscopy for live cell applications and technical inspection,” Appl. Opt. 47(4), A52–A61 (2008). [CrossRef]  

9. JW. Goodman, “Speckle phenomena in optics: theory and applications,” Roberts and Company Publishers (2007).

10. Y. Zhao, L. Cao, H. Zhang, and Q. He, “Holographic display with LED illumination based on phase-only spatial light modulator,” Information Optics and Optical Data Storage II. International Society for Optics and Photonics, 8559, 85590B (2012).

11. T. Nomura, M. Okamura, E. Niranai, and T. Numata, “Image quality improvement of digital holography by superposition of reconstructed images obtained by multiple wavelengths,” Appl. Opt. 47(19), D38–D43 (2008). [CrossRef]  

12. R. Guo, B. Yao, P. Gao, J. Min, M. Zhou, J. Han, X. Yu, X. Yu, M. Lei, S. Yan, Y. Yang, D. Dan, and T. Ye, “Off-axis digital holographic microscopy with LED illumination based on polarization filtering,” Appl. Opt. 52(34), 8233–8238 (2013). [CrossRef]  

13. Y. Takaki and M. Yokouchi, “Speckle-free and grayscale hologram reconstruction using time-multiplexing technique,” Opt. Express 19(8), 7567–7579 (2011). [CrossRef]  

14. Y. Mori, T. Fukuoka, and T. Nomura, “Speckle reduction in holographic projection by random pixel separation with time multiplexing,” Appl. Opt. 53(35), 8182–8188 (2014). [CrossRef]  

15. J. Amako, H. Miura, and T. Sonehara, “Speckle-noise reduction on kinoform reconstruction using a phase-only spatial light modulator,” Appl. Opt. 34(17), 3165–3171 (1995). [CrossRef]  

16. W. Hsu and C. Yeh, “Speckle suppression in holographic projection displays using temporal integration of speckle images from diffractive optical elements,” Appl. Opt. 50(34), H50–H55 (2011). [CrossRef]  

17. D. Yin, Z. Gu, Y. Zhang, F. Gu, S. Nie, S. Feng, J. Ma, and C. Yuan, “Speckle noise reduction in coherent imaging based on deep learning without clean data,” Opt. Lasers Eng. 133, 106151 (2020). [CrossRef]  

18. C. Chang, J. Xia, L. Yang, W. Lei, Z. Yang, and J. Chen, “Speckle-suppressed phase-only holographic three- dimensional display based on double-constraint Gerchberg-Saxton algorithm,” Appl. Opt. 54(23), 6994–7001 (2015). [CrossRef]  

19. H. Pang, J. Wang, A. Cao, and Q. Deng, “High-accuracy method for holographic image projection with suppressed speckle noise,” Opt. Express 24(20), 22766–22776 (2016). [CrossRef]  

20. H. Pang, J. Wang, M. Zhang, A. Cao, L. Shi, and Q. Deng, “Non-iterative phase-only fourier hologram generation with high image quality,” Opt. Express 25(13), 14323–14333 (2017). [CrossRef]  

21. L. Chen, S. Tian, H. Zhang, L. Cai, and G. Jin, “Phase hologram optimization with bandwidth constraint strategy for speckle-free optical reconstruction,” Opt. Express 29(8), 11645–11663 (2021). [CrossRef]  

22. T. Shimobaba and T. Ito, “Random phase-free computer-generated hologram,” Opt. Express 23(7), 9549–9554 (2015). [CrossRef]  

23. P. W. M. Tsang, Y. T. Chow, and T. C. Poon, “Generation of patterned-phase-only holograms (PPOHs),” Opt. Express 25(8), 9088–9093 (2017). [CrossRef]  

24. M. L. Cruz, “Full image reconstruction with reduced speckle noise, from a partially illuminated Fresnel hologram, using a structured random phase,” Appl. Opt. 58(8), 1917–1923 (2019). [CrossRef]  

25. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015). [CrossRef]  

26. D. Shen, G. Wu, and H. Suk, “Deep Learning in Medical Image Analysis,” Adv. Exp. Med. Biol. 19, 221–248 (2017). [CrossRef]  

27. A. Krizhevsky, I. Sutskever, and G. Hinton, “ImageNet classification with deep convolution neural networks,” Commun. ACM 60(6), 84–90 (2017). [CrossRef]  

28. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” International Conference on Medical image computing and computer-assisted intervention, (Springer, Cham,2015), Vol.9351, pp. 234–241.

29. P. Li, Z. Zhang, Q. Xiong, B. Ding, J. Hou, D. Luo, Y. Rong, and S. Li, “State-of-health estimation and remaining useful life prediction for the lithium-ion battery based on a variant long short term memory neural network,” J. Power Sources 459, 228069 (2020). [CrossRef]  

30. W. Jeon, W. Jeong, K. Son, and H. Yang, “Speckle noise reduction for digital holographic images using multi-scale convolutional neural networks,” Opt. Lett. 43(17), 4240–4243 (2018). [CrossRef]  

31. D. Y. Youl and J. H. Park, “Hologram conversion for speckle free reconstruction using light field extraction and deep learning,” Opt. Express 28(4), 5393–5409 (2020). [CrossRef]  

32. A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” Proceedings of the 30th International Conference on Machine Learning30(1), 3 (2013).

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1.
Fig. 1. The brief architecture of the neural network structure. The blocks mean the data matrixes. The red hollow arrow means the linear normalization operation. The red arrows mean the convolution, Leak ReLu activation function and a batch normalization (BN) operation. The blue arrows mean the transposed convolution, Leak ReLu activation function and BN operation. The red dash arrows mean the concatenate operation. The green arrow means the transposed convolution, tanh activation function and linear normalization operation. The black lines mean the data transmission and the black dash arrow means the data feedback to the proposed neural network.
Fig. 2.
Fig. 2. The target images, (a) “Square”, (b) “Football”, and (c) “Cartoon Girl”. The concern domains are marked with the yellow dashed boxes.
Fig. 3.
Fig. 3. The simulation results of the image “Square”; (a)-(c) The phase hologram, the reconstructed image, and the reconstructed phase distribution by GS algorithm (First row), DCGS algorithm (Second row), and our proposed method (Third row); (d) The cross-sectional diagram of the phase distribution along the dashed line in (c); (e) The cross-sectional diagram of the normalized intensity of the reconstructed images along the dashed lines in (b); (f) The graph of the loss function during training.
Fig. 4.
Fig. 4. The simulation results of the image “Football”; (a)-(c) The phase hologram, the reconstructed image, and the reconstructed phase distribution by GS algorithm (First row), DCGS algorithm (Second row), and our proposed method (Third row); (d) The cross-sectional diagram of the normalized intensity of the reconstructed images along the dashed lines in (b); (e) The graph of the loss function during training.
Fig. 5.
Fig. 5. The simulation results of the image “Cartoon Girl” ; (a)-(c) The phase hologram, the reconstructed image, and the reconstructed phase distribution by GS algorithm (First row), DCGS algorithm (Second row), and our proposed method (Third row); (d) The cross-sectional diagram of the phase distribution along the dashed line in (c); (e) The cross-sectional diagram of the normalized intensity of the reconstructed images along the dashed lines in (b); (f) The graph of the loss function during training.
Fig. 6.
Fig. 6. (a) The schematic diagram of the optical setup for the experiment; and the experimental results of the reconstructed image by GS algorithm (b); DCGS algorithm (c); and our proposed method (d).
Fig. 7.
Fig. 7. (a)-(e) The experimental results of the reconstructed images using GS algorithm (First row) and our proposed method (Second row) at z = 110 mm, 115 mm, 120 mm, 125 mm, 130 mm, respectively. The calculated PCC values are also shown in the lower left corner of each image.

Tables (1)

Tables Icon

Table 1. The PSNR of the reconstructed images with three different methods

Equations (13)

Equations on this page are rendered with MathJax. Learn more.

N = 2 G ,
{ f ( μ ) = t a n h ( μ ) + 1 2 t a n h ( u ) = e μ e μ e μ + e μ ,
U ( x , y , z ) = F 1 { F { exp [ j π φ ( x , y ) ] } exp [ j 2 π z 1 λ 2 f x 2 f y 2 ] } ,
f x = f y = n L ,
{ U m , n = a m , n + b m , n i A m , n = a m , n 2 + b m , n 2 P m , n = tan 1 ( b m , n a m , n ) ,
L a = m , n = 1 N ( A m , n A ¯ ) ( y m , n y ¯ ) m , n = 1 N ( A m , n A ¯ ) 2 m , n = 1 N ( y m , n y ¯ ) 2 ,
Δ P m , n r = { P m + 1 , n P m , n , ( m = 1 , 2 , , N 1 ; n = 1 , 2 , , N ) 0 , ( m = N ; n = 1 , 2 , , N ) ,
{ Δ P m , n r _ w = B ¯ , ( w h e r e y m , n = 0 ) Δ P m , n r _ w = Δ P m , n r , ( w h e r e y m , n 0 ) ,
{ Δ P r _ w ¯ = α B ¯ + β B ¯ N × N α + β = N 2 ,
Δ P r _ w ¯ = B ¯ ,
Δ P s t d r _ w = 1 N m , n N ( Δ P m , n r _ w Δ P r _ w ¯ ) 2 N × N ,
L p = Δ P s t d r _ w + Δ P s t d c _ w 2 ,
T = ω T a + ( 1 ω ) T p ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.