1. Introduction

Holographic data storage (HDS) is expected to be applied to storage of the ever-increasing quantities of digital data around the world, especially the cold data in data centers, and will offer benefits such as high speed, large storage capacity, and low power consumption for data retention [1–4]. In the efforts to provide further improvements in the data transfer rate and the recording capacity of HDS, both the conventional method of modulating the intensity distribution of the signal beam and the method of modulating the signal beam’s phase distribution have attracted attention [5–8]. Furthermore, the complex amplitude modulation technique, in which spatial modulation is applied to both the intensity distribution and the phase distribution of the signal beam, is currently being studied actively [9–15]. We have focused on complex amplitude modulation of the signal beam and its detection as elemental techniques that are not considered as part of conventional intensity-modulation-based HDS and that must be newly developed to realize complex-amplitude-modulation-based HDS. In particular, we have investigated the transport of intensity equation (TIE) method, which is not reliant on bulky interferometric optics, as a vibration-tolerant technique for detection of the complex-amplitude-modulated signal beam [16,17]. During detection of the complex amplitudes of light waves using the TIE method, phase noise with a characteristic cloud-like spatial distribution that is caused by optical and electrical noise and alignment errors is known to be superimposed on the detected phase distribution [18,19]. If this noise value is relatively small, it can then be estimated and removed using the linear interpolation method that we proposed in [16], and the data page can be decoded accurately. However, when the phase noise is higher than a certain level, this noise estimation/removal approach will not work properly and the data page decoding accuracy may be degraded significantly. As a result, the number of symbols per data page and the numbers of intensity and phase modulation levels are limited.

Realization of the TIE method, which normally requires multi-shot acquisition of through-focus images, using a single-shot acquisition method is useful in practical terms because it enables the measurement of dynamic objects during quantitative phase imaging. Therefore, several single-shot methods have been proposed to date [20–24]. In this paper, we apply a single-shot TIE method that uses two cameras as a method for detection of the complex-amplitude-modulated signal beam in HDS for the first time. Introduction of this technology to HDS will improve the data transfer rates dramatically by enabling data playback with a single signal intensity acquisition per complex-amplitude-modulated data page. However, the single-shot TIE method tends to suffer from stronger characteristic cloud-like noise than the multi-shot TIE method, which is implemented by translating a single camera using a motorized stage or by placing a variable focal length lens in the optical path of the light to be measured. The origin of this higher noise is that the alignment errors of the optical elements, including the spatial light modulator (SLM) and the two cameras, and the difference between the two light intensities, which are split using a beam splitter (BS) placed in front of the cameras, both increase.

To overcome this drawback, in this paper, we propose a method for accurate classification and decoding of complex-amplitude-modulated data pages, which are detected not only by the multi-shot TIE method but also by the single-shot TIE method using simple convolutional neural network (CNN)-based classifiers. We demonstrate experimentally that the classifiers can both classify and decode the multivalued intensity- and phase-modulated data pages, even when the detected phase is heavily contaminated by the cloud-like noise that typically occurs with the TIE method. Recently, the introduction of neural network technology into HDS has been studied actively [25–30]. In [29], the classification and decoding of complex-amplitude-modulated data pages using a CNN-based classifier was also studied. A modulation code was proposed to enable the complex-amplitude-modulated data page to achieve low-error-rate decoding, and CNN-based classifiers that were suitable for data page demodulation were proposed, with their usefulness being demonstrated through numerical simulations. In these simulations, the complex amplitude of the signal beam was detected using the well-known phase-shifting method and appropriate noise levels for the simulation conditions were taken into account. In contrast, the phase noise superimposed on the phase detected via the TIE method shows a characteristic cloud-like spatial distribution and can have phase values that are large enough to exceed the modulation phase greatly; this is a completely different situation to the noise associated with other complex amplitude detection methods for light waves. Therefore, a completely novel investigation is inevitably required to realize highly accurate decoding of the complex-amplitude-modulated data pages detected using the TIE method.

The remainder of this paper is organized as follows. Section 2 outlines the approach for detection of the complex-amplitude-modulated signal beam in HDS using the TIE method including the typical cloud-like noise that is superimposed on the detected phase and a method for its estimation and removal. In this work, we newly apply a method for detection of the complex-amplitude-modulated signal beam based on a single-shot TIE method that uses two cameras to HDS. Section 3 describes a classification and decoding method based on CNN-based classifiers for the intensity and phase data pages in the complex-amplitude-modulated signal beam detected via the TIE method. First, we describe the modulation format for the intensity and phase data pages and the preparation of the datasets used to train, validate, and test the classifiers. Then, we explain the neural network architecture used in this study in detail. In Section 4, we verify experimentally that the CNN-based classifiers can be used to perform accurate classification and decoding of the multivalued intensity and phase data pages in the complex-amplitude-modulated signal beam when detected not only by the multi-shot TIE method but also by the single-shot TIE method. Finally, Section 5 presents the conclusions drawn from this paper.

2. Detection of a complex-amplitude-modulated signal beam with the TIE method in HDS

The TIE is the equation below, which describes the relationship between the change in the intensity distribution of a light wave caused by propagation and the phase distribution of that light wave [31].

Figure 1 shows a conceptual diagram for HDS based on detection of the complex-amplitude-modulated signal beam using the TIE method. In this figure, the complex signal beam amplitude is assumed to be modulated via a computer-generated hologram (CGH) technique using a phase-only SLM, as described in detail in [17]. An interference fringe formed by the signal beam intersecting with a reference beam is recorded as a hologram in the holographic material and the recorded signal beam can then be reproduced when the hologram is irradiated by the reference beam. When the multi-shot TIE method is used to detect the complex-amplitude-modulated signal beam, only camera 1 in the figure is used. When using the differential approximation based on the central difference, the phase distribution of the light wave can be determined by acquiring the light intensity distributions at $z = -\Delta z, 0$, and $\Delta z$, substituting $\partial I(x,y,0) / \partial z = \left \{ I(x,y,\Delta z) - I(x,y,-\Delta z)\right \}/ (2 \Delta z)$ and the in-focus image $I(x,y,0)$ into equation (1), and then solving for $\phi (x,y,0)$. When using the differential approximation based on the forward difference, the phase distribution of the light wave can be determined by acquiring the light intensity distributions at $z = 0$ and $\Delta z$, substituting $\partial I(x,y,0) / \partial z = \left \{ I(x,y,\Delta z) - I(x,y,0)\right \}/ \Delta z$ and the in-focus image $I(x,y,0)$ into Eq. (1), and then solving for $\phi (x,y,0)$.

 

Fig. 1. Conceptual diagram of HDS using complex amplitude modulation with a single phase-only SLM and signal beam detection performed by the TIE method. Camera 1 alone is used in the multi-shot TIE method, while camera 1 and camera 2 are used simultaneously in the single-shot TIE method.

Download Full Size | PDF

Two cameras are used to detect the complex-amplitude-modulated signal beam synchronously in the single-shot TIE method. Camera 1 is placed at the in-focus position of $z = 0$ in the optical path of the transmitted beam at the BS and camera 2 is placed at the defocus position of $z = \Delta z$ in the optical path of the reflected beam at the BS. The subsequent procedure is same as that used in the multi-shot TIE method when using differential approximation based on forward differencing. The complex amplitude detection accuracy of the single-shot TIE method should ideally be equivalent to that of the multi-shot TIE method when using the forward difference approximation. However, in actual optical systems, the accuracy of complex amplitude detection when using the single-shot TIE method tends to be lower than that for the multi-shot TIE method because of the imperfect alignment of the two cameras and the mismatched intensities of the transmitted and reflected beams at the BS. In this study, we have attempted to compensate for this intensity discrepancy by adjusting the exposure times for the two cameras separately. Although in reality it was difficult to balance the intensities of the two images acquired by two cameras perfectly, this adjustment method worked reasonably well.

3. Classification and decoding of data pages in the complex-amplitude-modulated signal beam detected by the TIE method using CNN-based classifiers

In this section, we propose a method to classify and decode multivalued intensity and phase data pages in the complex-amplitude-modulated signal beam when detected via the TIE method using CNN-based classifiers. This method specifically enables accurate classification and decoding of multivalued phase data pages without removal of the large cloud-like phase noise characteristic of the TIE method that is superimposed on the detected phase, thus enabling single-shot TIE detection of the complex-amplitude-modulated signal beam.

3.1 Data page format and datasets used for training and evaluation of the classifiers

This subsection introduces the data page format used in this study and the datasets that were used to train and evaluate the classifiers. As Fig. 2(a) shows, the data page in the complex-amplitude-modulated signal beam consists of $16 \times 16$ symbols, where each of these symbols is randomly modulated to have one of two intensity values and one of four phase values ($\pi /2, \pi, 3\pi /2$, and $2\pi$ rad).

 

Fig. 2. Conceptual diagram of the complex amplitude modulation applied to the signal beam and extraction of the input data to the classifiers from the signal beam detected using the TIE method. (a) Intensity and phase modulation format of data pages in the signal beam, (b) symbol blocks, (c) intensity and phase distributions detected using the TIE method, and (d) training/validation/test data extracted from the distributions in (c).

Download Full Size | PDF

As described in greater detail in [16], the TIE method requires spatial continuity within the phase to be detected, and thus the phase within a symbol is designed to vary continuously in the form of a sine wave, with a phase value of zero at the boundaries with adjacent symbols. During classification and decoding of the symbols in a data page using the classifiers, we treat $2 \times 2 = 4$ symbols as a single symbol block, as shown in Fig. 2(b). An intensity symbol block has $2^4 = 16$ classes based on binary amplitude modulation and a phase symbol block has $4^4 = 256$ classes based on quadrature phase modulation. As shown in Fig. 2(a), the intensity and phase data pages consist of arrays of the symbol blocks for the intensity and phase information, respectively, with the blocks being arranged in two dimensions in both cases. In an HDS, a signal beam that is subjected to such amplitude and phase modulation, or to complex amplitude modulation, is recorded in the holographic material and can then be reproduced from this material and detected via the multi-shot or single-shot TIE methods, as shown in Fig. 2(c). The detected intensity and phase data pages in the signal beam are separated into symbol blocks that consist of $2 \times 2$ symbols, as shown in Fig. 2(d), and are then used as training, validation, and test data in the CNN-based classifiers, along with labels that represent the correct classes of these symbol blocks. As Fig. 2(c) shows, the phase distribution detected using the TIE method is superimposed with characteristic cloud-like noise; this noise will be discussed in detail in Section 4. In this study, a phase distribution from which this cloud-like noise is not removed but is instead left intact is used as the input to the classifier.

3.2 Network architecture

Figure 3 shows the architecture of the CNN-based classifiers used in this study. Note that this figure represents two neural networks, where one is used for the intensity symbol blocks and the other is used for the phase symbol blocks, with different numbers of outputs in their final layers. Note also that these two networks are trained and tested using independent processes.

 

Fig. 3. Architecture of the CNN-based classifiers used in this study. Conv: convolutional layer; Pool: max pooling layer; Do: dropout layer; FC: fully connected layer.

Download Full Size | PDF

These networks were built using the Keras framework [32]. The architecture consists of four convolutional layers, two fully connected layers, two max pooling layers, and two dropout layers. The kernel size of the filters and the numbers of filters in the convolutional layers are $3 \times 3$ and 32, respectively, as indicated in Fig. 3. In the convolutional layers, when sliding the filter, a sum-of-products operation (i.e., a convolution process) is performed between the filter coefficients and the input signal, and the result is then output [33]. Fully connected layer FC1 consists of 128 nodes, while fully connected layer FC2 contains 16 nodes for classification and decoding of the intensity symbol blocks, and 256 nodes for classification and decoding of the phase symbol blocks. These numbers are consistent with the numbers of classes for the intensity and phase symbol blocks. The rectified linear unit (ReLU) function was used as the activation function for the convolutional layers and for the first fully connected layer FC1, and the Softmax function was used as the activation function for the second fully connected layer FC2. The output from FC2 indicates the probability that a symbol block input to the classifiers belongs to each class. The class that has the highest probability is then used as the classification result. In the max pooling layer, a $2 \times 2$ window is slid both horizontally and vertically, with only the largest values in the window being extracted. A feature map is then output in the form of a matrix that is half the size of the input for the layer in both the vertical and horizontal directions. The dropout layer suppresses overfitting of classifiers to the training data by deactivating a certain percentage of the nodes during neural network training [34]. The dropout rates were set at 0.5 for Do1 and 0.25 for Do2. These neural networks for the intensity and phase were given the intensity and phase symbol blocks and labels shown in Fig. 2(d) to perform training, validation, and testing. Here, the categorical cross-entropy and the accuracy were used as the loss function and the evaluation index, respectively, during training, validation, and testing. The filter coefficients and weights of the convolutional layers and the weights and biases of the fully connected layers were initialized to have random values, and during training, these parameters were optimized to minimize the loss function. We used back-propagation and an Adam optimizer with the learning rate of 0.1. The batch size was set at 128.

4. Experimental demonstration

In this section, we show experimentally that the CNN-based classifiers of the type illustrated in Fig. 3 can be used to classify and decode the intensity and phase data pages in the complex-amplitude-modulated signal beam accurately when the beam is detected via the TIE method. Furthermore, we verify that this technique also enables single-shot TIE detection of the complex- amplitude-modulated signal beam in HDS.

4.1 TIE detection of the complex-amplitude-modulated signal beam and preparation of datasets

Figure 4 shows the experimental setups with and without hologram recording and reconstruction. A 532 nm diode-pumped solid-state laser (Excelsior-532-200, Spectra-Physics, Inc., USA) was used as the light source and the HSP1920-500-1200-HSP8 (Meadowlark Optics Inc., USA) and the ORCA-Flash4.0 V3 (Hamamatsu Photonics K.K., Japan) were used as the phase-only SLM and the complementary metal-oxide-semiconductor (CMOS) camera model, respectively. In Fig. 4(a), a complex amplitude modulation method for the signal beam was applied that was originally proposed in [35]. In this method, the signal beam phase is modulated by a single phase-only SLM that displays a CGH with a linear phase grating. The modulated signal beam is then Fourier-transformed by the first lens. At the back focal plane of this lens, the first-order diffracted beam alone is extracted via an aperture. The extracted beam is then inverse-Fourier-transformed by the second lens and the desired complex amplitude is realized at the back focal plane of the lens. Application of this technique to HDS was discussed in detail previously in [17]. When hologram recording and reconstruction are performed, the variations in the diffracted light intensity and the degradation of the reconstructed images make it difficult to evaluate the classification and decoding performance of the CNN-based classifiers purely for the complex-amplitude-modulated signal beam detected using the TIE method. Therefore, for the majority of the experiments, hologram recording and retrieval were not performed, and the signal beam generated using the SLM, the lenses, and spatial filtering was detected directly via the TIE method. Then the symbol blocks in the beam were classified/decoded via the classifiers using the experimental setup shown in Fig. 4(a). In particular, investigation of the influence of the cloud-like phase noise on the classification and decoding performance is the main purpose of this study. However, in the final part of the next subsection, the performance of the CNN-based classifier that was trained using the datasets without hologram recording/retrieval was also confirmed by classifying and decoding phase symbol blocks that were extracted from a phase data page that had been reconstructed from a hologram recorded in a photopolymer material using the experimental setup shown in Fig. 4(b). This result verifies that this classification method continues to be effective even when the signal beam is reconstructed from a hologram recorded in holographic material.

 

Fig. 4. Experimental setups (a) with and (b) without hologram recording and reconstruction. HWP: half-wave plate; BS: beam splitter; PBS: polarizing beam splitter; L: lens; M: mirror.

Download Full Size | PDF

In the following, we describe the preparation of the training, validation, and test data for the CNN-based classifiers. In the experiments, we investigated the accuracy of the classification and decoding operations performed using the classifiers for three TIE method types: a multi-shot TIE method (Type 1) that used three intensity images acquired by camera 1 and the central difference method; another multi-shot TIE method (Type 2) that used two intensity images acquired by camera 1 and the forward difference method; and a single-shot TIE method (Type 3) that used two intensity images acquired by cameras 1 and 2 and the forward difference method. For all cases ranging from Type 1 to Type 3, 500 intensity- and phase-modulated random data pages composed of $16 \times 16$ symbols were generated, and the complex amplitude of the signal beam was modulated sequentially according to these data pages. Each symbol in the data pages consisted of $28 \times 28$ SLM pixels, and each symbol was randomly modulated to have one of two intensity values and one of four phase values ($\pi /2, \pi, 3\pi /2$, and $2\pi$ rad). The intensity and phase distributions of these signal beams were then detected using each of the TIE methods as follows. Camera 1 was moved to positions of $z = -\Delta z, 0$, and $\Delta z$ for the Type 1 method and positions of $z = 0$ and $\Delta z$ for the Type 2 method using a motorized stage installed below camera 1, and the signal beam intensity distributions corresponding to the 500 data pages were then captured at their respective z-coordinates. In the Type 3 method, the intensity distributions of the signal beam corresponding to the 500 data pages were captured synchronously by both camera 1 and camera 2, which were positioned at $z = 0$ and $z = \Delta z$, respectively. Next, the complex amplitudes of the signal beam corresponding to the 500 data pages were calculated using each TIE method. As shown in Fig. 2(d), the symbol blocks for the intensity and the symbol blocks for the phase, which consisted of $2 \times 2$ symbols, were extracted from the intensity and phase data pages. The minimum phase value in each phase symbol block was preprocessed to be zero degrees by subtracting the minimum value within each phase symbol block from the value of each phase symbol block. Then, the values of the phase symbol blocks were divided by 360. The values of the intensity symbol blocks were divided by 65,536, which is the maximum pixel value of the cameras. Because $(16/2) \times (16/2) = 64$ symbol blocks were obtained per data page, the dataset for the 500 data pages consisted of $64 \times 500 = 32,000$ symbol blocks for the intensity and phase. From the total number of 32,000 symbol blocks for each intensity and phase dataset, 3,200 blocks were used as test data, 25,920 of the remaining 28,800 blocks were used as training data, and 2,880 blocks were used as validation data during training. In the experiments, the size of one symbol block in the camera plane was $91 \times 91$ camera pixels, which is the input size to the classifiers, as indicated in Fig. 3.

Figure 5 shows examples of the intensity images acquired by the cameras to enable detection of the signal beam when modulated by the first complex-amplitude-modulated data page of the total of 500 pages. Figure 5(a) shows the three intensity images captured by camera 1 to enable the multi-shot TIE methods to be performed, where $\Delta z$ is 3 mm. All three images were used to perform the Type 1 method, and the two images at $z = 0$ mm and 3 mm were used to perform the Type 2 method. Figure 5(b) shows the intensity image captured at $z = 0$ mm by camera 1 and the corresponding image captured at $z = 10$ mm by camera 2, which were required to perform the Type 3 method, i.e., the single-shot TIE method. The ratio of the sum of the pixel value counts of the reflected signal beam at the BS in Fig. 4 to that of the transmitted signal beam was 0.736 when both cameras were placed in the in-focus positions. The exposure times of 25 ms for camera 1 for the transmitted signal beam and 35 ms for camera 2 for the reflected beam were selected from the available exposure times to achieve a ratio of 0.714, which is close to 0.736. This procedure compensates for the discrepancy between the transmittance and the reflectance of the BS.

 

Fig. 5. Signal beam intensity images acquired by the cameras to detect the first complex-amplitude-modulated data page using the three TIE methods. (a) Intensity images used for the multi-shot TIE methods. (b) Intensity images used for the single-shot TIE method. The image at $z = 10$ mm is horizontally inverted because of the reflection that occurred at BS2.

Download Full Size | PDF

In the TIE method under ideal conditions, when a small $\Delta z$ is used, the error caused by the difference approximation of the z-differential of the intensity distribution is reduced, and this contributes to improvement of the phase measurement accuracy. However, in a realistic situation, the changes in the intensity distributions before and after propagation over the small $\Delta z$ become small, and the differences between the intensity distributions before and after propagation also only have small values, which makes the z-differential of the intensity distribution sensitive to various noise sources and errors introduced via the intensity acquisition. These noise sources and errors can then be a major cause of the strong cloud-like noise. In the experiments conducted in this study, three different differential approximation methods were used, where the $\Delta z$ required to reduce the cloud-like noise was different for each method. We tried various $\Delta z$ values for each method and found that the $\Delta z$ values shown in Fig. 5 yielded a reasonably small amount of the cloud-like noise.

Figure 6 shows the phase distributions of the complex-amplitude-modulated signal beam detected via the Type 1, Type 2, and Type 3 TIE methods when using the intensity distributions shown in Fig. 5, along with the corresponding phase distributions after noise removal was performed using the linear interpolation method; the related constellation diagrams of the symbols in the signal beam after noise removal are also shown. In the constellation diagram, the complex values of the $16 \times 16$ symbols are plotted on the complex plane. Each point in the constellation diagram is given a different color to represent each modulation value of the symbol. The phase distributions measured using the TIE method tend to be superimposed with characteristic cloud-like noise, as shown in Fig. 6(a-1), (a-2), and (a-3). This occurs because low-spatial-frequency components of the error in the approximated $\partial I / \partial z$ from the true value, which originates from scattered light noise, electrical noise in the camera, misalignment of the optical elements, and other sources, are amplified during the process of numerical solution of the TIE. Figure 6(b-1), 6(b-2), and 6(b-3) all used the noise removal method proposed in [16]. This method uses a simple linear interpolation process to estimate and remove the cloud-like noise based on the fact that the noise phase in space changes more slowly than the modulation phase in the data page. The noise is estimated based on the assumption that the noise varies linearly in the vertical direction in a symbol. Because the continuous modulation phase value is designed to be zero on boundaries between adjacent symbols, the phase value of the cloud-like noise in each symbol can be estimated via linear interpolation of the phase values at each symbol’s upper and lower boundaries. The estimated cloud-like noise was then subtracted from the detected TIE phase to remove the cloud-like noise. Although the modulation phase ranges from 0 to 360 degrees, as shown in Fig. 2(a), the dynamic range of the detected phase is considerably greater than that range, which indicates the strong influence of the cloud-like noise on the detected TIE phase. This undesirable expansion of the phase dynamic range increases from Fig. 6(a-1) to (a-3) and subsequently causes increases in the variance of each symbol plot in the constellation diagrams, even after noise removal, as illustrated in Fig. 6(c-1), (c-2), and (c-3). Figure 6(c-3) shows that it is no longer possible to identify the groups of symbols independently in the constellation diagram, even after noise removal, when the complex-amplitude-modulated signal beam is detected using the single-shot TIE method. This occurs because, when the strength of the cloud-like noise increases, the assumption that the noise is spatially linear in each symbol no longer holds, and the simple denoising technique then cannot remove the noise effectively. In some cases, it may be possible to decode the data pages through skillful use of block coding, error correction, or other processes. However, it is impossible to decode the data pages without error when using a simple hard decision method based on threshold values.

 

Fig. 6. Phase distributions and constellation diagrams for the complex-amplitude-modulated signal beams detected using the Type 1, Type 2, and Type 3 TIE methods. (a-1)–(a-3) Phase distributions of the signal beams when detected using the three TIE methods. (b-1)–(b-3) Phase distributions of the signal beams in (a-1)–(a-3) following noise removal, respectively. (c-1)–(c-3) Constellation diagrams of the $16 \times 16$ symbols plotted with the intensity distributions at $z = 0$ mm in Fig. 5 and the phase distributions from (b-1)–(b-3), respectively.

Download Full Size | PDF

4.2 Classification and decoding of intensity and phase symbol blocks using the CNN-based classifiers

In this subsection, classification and decoding of both the binary-modulated intensity symbol blocks and the quadrature-modulated phase symbol blocks were attempted using the CNN-based classifiers. Note that no techniques for removal of the cloud-like noise from the phase distribution were used in this experiment.

Figure 7(a-1), (a-2), and (a-3) show the losses for the epoch in the training and validation case when the phase symbol blocks generated by the three TIE methods (i.e., Type 1 to Type 3) were used as the datasets. Figure 7(b-1), (b-2), and (b-3) then show the corresponding accuracy characteristics for the epoch for the three methods, respectively. Finally, Fig. 7(c-1), (c-2), and (c-3) show the corresponding heatmap representations of the confusion matrix and the accuracy of the test data at the end of 100 epochs for the three methods, respectively. Figure 7(a-1)–7(a-3) show that for all method types, the loss decreases until it approaches zero as the epoch progresses, whereas the accuracy improves until it approaches unity as the epoch progresses as shown in Fig. 7(b-1)–7(b-3). The speed of improvement in both the loss and the accuracy characteristics for the Type 3 method is slower than the corresponding speeds for the Type 1 and Type 2 methods. However, through repeated optimization of the filter coefficients, weights, and biases of the network over a sufficient number of epochs, good loss and accuracy values can even be achieved for the Type 3 method. The vertical axis for the heatmaps shown in Fig. 7(c-1), 7(c-2), and 7(c-3) represents the class of the phase symbol block that was input to the classifier, and the horizontal axis represents the class into which the symbol block was classified by the classifier. The colors in the heatmap represent the number of occurrences of each classification. These figures confirm that the input phase symbol blocks were classified accurately into the correct 256 classes for all TIE method types. As the bottom section of Fig. 7 shows, classification accuracies of 100% were obtained for the test data after 100 epochs for all TIE method types, confirming the high classification/decoding performances of the three methods.

 

Fig. 7. Classification and decoding results obtained when using the CNN-based classifier for the phase symbol blocks extracted from the phase distributions detected via the Type 1, Type 2, and Type 3 TIE methods. (a-1)–(a-3) Training and validation loss characteristics for the epoch. (b-1)–(b-3) Training and validation accuracy characteristics for the epoch. (c-1)–(c-3) Confusion matrix and test accuracy characteristics.

Download Full Size | PDF

Figure 8 shows the results of classification and decoding of the intensity symbol blocks that were extracted from the in-focus intensity images for the Type 3 TIE method when using the CNN-based classifier. The classification results obtained for the three TIE methods from Type 1 to Type 3 were almost identical in this experiment, as expected, because the in-focus intensity distribution captured directly by camera 1 was used commonly to generate the intensity symbol blocks. Figure 8(a)–(c) show the test and validation loss characteristics for the epoch, the accuracy characteristics for the epoch, and the confusion matrix after 50 epochs, respectively. Both the loss value and the accuracy converged rapidly, and after 50 epochs, the intensity symbol blocks were classified correctly and were decoded into 16 classes without error.

 

Fig. 8. Classification and decoding results obtained by the CNN-based classifier for the intensity symbol blocks extracted from the in-focus intensity images for the Type 3 TIE method. (a) Training and validation loss characteristics for the epoch. (b) Training and validation accuracy characteristics for the epoch. (c) Confusion matrix.

Download Full Size | PDF

When the phase symbol block shown in Fig. 9(a) was input to the classifier that had been trained with the dataset obtained using the Type 3 TIE method, the four convolutional layers then output the feature maps shown in Fig. 9(b). Although cloud-like noise was observed in the feature maps of the first convolutional layer, the noise in the feature maps was reduced in the deeper layers, and the outputs at the four symbol positions in the symbol block increase. Therefore, it appears that the network does tend to be able to ignore the cloud-like noise in the deeper layers and acquire the features of the modulation phase in the symbol blocks. This may be the reason why the CNN-based classifier is able to classify and decode phase symbol blocks with high cloud-like noise accurately.

 

Fig. 9. Output feature maps obtained from convolutional layers for a phase symbol block input. (a) Input phase symbol block obtained using the Type 3 TIE method ($\Delta z = 10$ mm. Top left symbol block in the 250th data page from the total of 500 pages). (b) Output feature maps obtained from each convolutional layer.

Download Full Size | PDF

The phase distributions detected using the Type 3 TIE method for various $\Delta z$ values are shown in Fig. 10(a). In the figure, the cloud-like noise increases as $\Delta z$ decreases, which can be understood from the perspective of the visibility of the phase signal modulation versus the cloud-like noise and the dynamic range of the color bar. Figure 10(b) shows the $\Delta z$ dependence of the classification and decoding accuracy of the phase symbol blocks when processed by the CNN-based classifier. In this figure, the horizontal axis shows the $\Delta z$ value used in the TIE method to obtain the dataset for testing. In other words, this horizontal axis corresponds roughly to the opposite of the amount of cloud-like noise contained in the test dataset. The results are shown for the cases in which the network was trained using the phase datasets generated by the Type 3 TIE method when using: I: $\Delta z$ equal to the value of $\Delta z$ for testing indicated by the horizontal axis of the graph; II: $\Delta z = 4$ mm; III: $\Delta z =$ 4 and 6 mm; and IV: $\Delta z =$ 3, 4, and 6 mm. In cases III and IV, the datasets were obtained by merging the datasets with their respective $\Delta z$ values. The results for case I show that the classifier is able to classify the phase symbol blocks accurately when the noise contained in the training data is statistically comparable to the noise contained in the test data; this holds even in the presence of the cloud-like noise that is considerably greater than the 0–360 degrees modulation phase. The results for case II show that the classifier exhibits a certain degree of tolerance even when the $\Delta z$ (or the amount of noise) for the test dataset is different to the $\Delta z$ (or the amount of noise) for the training dataset. The results of cases III and IV also show that this tolerance can be controlled and extended using a training dataset composed of merged datasets with different $\Delta z$ values. The merged datasets include a wider variety of values, and further performance improvements may be achieved by increasing the number of parameters and/or the numbers of layers within the network.

 

Fig. 10. Detected phase distributions and test accuracy characteristics of the CNN-based classifier for the $\Delta z$ values used in the TIE method to obtain the test dataset. (a) Detected phase distributions for various $\Delta z$ values (250th data page of the total of 500 pages.). (b) Test accuracy relative to the $\Delta z$ value used to prepare the test dataset for the various training datasets.

Download Full Size | PDF

Finally, the CNN-based classifier was tested using phase symbol blocks that had been extracted from a data page in a signal beam that was reconstructed from a hologram recorded in a photopolymer (TG-003, Kyoeisha Chemical Co. Ltd., Japan) layer with a thickness of 400 µm. The phase symbol blocks used to perform the classification test were created using the experimental setup shown in Fig. 4(b). In Fig. 4(b), the pinhole shown in Fig. 4(a) was removed by inserting the photopolymer in its place. Therefore, phase-only modulation without the linear phase grating was applied to the signal beam in this experiment and the zeroth-order beam from the pixelated SLM was used as the signal beam to perform hologram recording. Holograms were recorded using the intersecting signal and reference beams, and the signal beam was then reproduced by irradiating the hologram with the reference beam. A diffuser located on the reference arm was used to eliminate any unwanted reconstructions that occurred from previously recorded holograms. Figure 11(a) shows the intensity images acquired using camera 1 at $\Delta z = 0$ mm and camera 2 at $\Delta z = 10$ mm, and Fig. 11(b) shows the phase distribution that was calculated from these images using the Type 3 TIE method with $\Delta z$ = 10 mm. Sixty-four pieces of the phase symbol blocks were generated from this phase distribution and were then used as the input test data for the CNN-based classifier that had been trained using datasets that had not experienced hologram recording and reconstruction. We used two different training datasets in this case. The first was the phase dataset created using the complex amplitude modulation optics shown in Fig. 4(a). This training dataset was the same dataset that had been used in the previous experiments. The second was the phase dataset that was newly created using the phase-only modulation optics shown in Fig. 4(b). The photopolymer shown in Fig. 4(b) was removed during training dataset preparation. In TIE detection of a signal beam that was reconstructed from a recorded hologram, the quantity of cloud-like noise and its distribution characteristics may differ when compared with the corresponding parameters in signal beams that were detected directly without hologram recording and reconstruction because of the scattering that occurs at the holographic material and the lower fidelity of the holographically reproduced images. We therefore prepared training datasets with a variety of $\Delta z$ values (i.e., with various depths of cloud-like noise) and adopted a dataset that provided good classification results. The classification accuracies for the test phase symbol blocks generated from the signal beam that had been reconstructed from the recorded hologram were 46.9% for the classifier that was trained using the dataset prepared with the complex amplitude modulation optics, and 62.5% for the classifier that was trained using the dataset prepared with the phase-only modulation optics. The training datasets prepared with $\Delta z$ = 7 mm were used in both cases. Figure 11(c) shows the confusion matrix for the classification that was performed with the classifier trained using the dataset prepared with complex amplitude modulation optics; Fig. 11(d) shows the corresponding confusion matrix for the classification that was performed with the classifier trained using the dataset prepared with phase-only modulation optics. Although the classification accuracies of 46.9% and 62.5% realized in these cases are not high enough to enable immediate practical use, these values still indicate that phase symbol blocks that have experienced hologram recording and reconstruction have the same characteristics as those obtained from datasets without hologram recording and reconstruction. Both confusion matrices show that the plots are concentrated near the diagonal. The figures also show that the off-diagonal points form several straight lines oriented parallel to the diagonal. This indicates that even when classification of a symbol block fails, only a small number of the four symbols contained within a single symbol block are wrong in many cases. The symbol decoding success rates can be calculated by examining the digit-by-digit agreement between the quaternary representations of the input and output symbol values in the classification results shown in Fig. 11(c) and 11(d), where the rates obtained were 83.2% and 87.1%, respectively. These rates have relatively high values for a proof-of-principle experiment and thus indicate the feasibility of phase symbol decoding from holographically reconstructed signal beams via the single-shot TIE method. Further work will be required in future to improve the classification and decoding accuracies for datasets with hologram recording and reconstruction by enhancing the hologram reconstruction fidelity, modifying the network architecture, and using symbol blocks that have experienced hologram recording and reconstruction as part of the network training procedure.

 

Fig. 11. Classification results produced by the CNN-based classifier for the phase symbol blocks in a signal beam reconstructed from a hologram in a photopolymer. (a) Intensity images acquired using cameras 1 and 2. The image acquired by camera 2 is horizontally inverted because of the reflection that occurred at BS2. (b) Phase distribution obtained by the Type 3 TIE method. (c) Confusion matrix for the classifier that was trained using the dataset prepared with the complex amplitude modulation optics. (d) Confusion matrix for the classifier that was trained using the dataset prepared with the phase-only modulation optics.

Download Full Size | PDF

5. Discussion and conclusions

In this paper, we have proposed a method for HDS to classify and decode the intensity and phase data pages contained in complex-amplitude-modulated signal beams that were detected via the TIE method using simple CNN-based classifiers, and have then demonstrated the usefulness of the proposed method experimentally. This method enables highly accurate classification and decoding of complex-amplitude-modulated data pages without the need to remove the cloud-like noise that typically appears in the phase distribution when detected using the TIE method. Furthermore, we applied a single-shot TIE method using two cameras to detection of the complex-amplitude-modulated signal beam in HDS. Although relatively high phase noise typically tends to be superimposed on any phase distributions detected via this method, it was possible to classify and decode the data pages in the complex-amplitude-modulated signal beam accurately when using the CNN-based classifier without removal of this phase noise. The $\Delta z$ value dependence of the classification accuracy was also investigated experimentally. The classifiers showed some classification accuracy tolerance to deviation of the $\Delta z$ value used in the test dataset preparation from the $\Delta z$ value used in training dataset preparation, and this tolerance was shown to be controllable by adjusting the $\Delta z$ value used during preparation of the training dataset. These results were obtained by direct detection of the signal beam that was modulated by the SLM without hologram recording and reconstruction. Finally, the phase symbol blocks in the signal beam that were reproduced from a hologram recorded in a photopolymer were used as the test data for the classifier. As a result, the phase symbol blocks and the four-valued symbols in these blocks, which have experienced hologram recording and reproduction, were classified and decoded with accuracies of 62.5% and 87.1%, respectively. The experimental results in this study provide comprehensive proof that the CNN-based classifier has demonstrated great potential for accurate classification and decoding of phase symbol blocks that have been strongly contaminated by cloud-like noise, and the classifier is even valid when the signal beam is reconstructed from a hologram recorded in holographic materials. The classification and decoding accuracies of the CNN-based classifiers for the signal beam reconstructed from the hologram may be improved by generating the training dataset from the data pages in the signal beam that have experienced hologram recording and reconstruction. The experimental results reported in this study verify the feasibility of high-speed, high-accuracy detection of complex-amplitude-modulated signal beams in HDS using the single-shot TIE method.

TIE detection of the complex-amplitude-modulated signal beam requires that the phase of the signal beam be modulated continuously, and at present, multiple pixels of the phase SLM are used to modulate each symbol. This causes an increase in the symbol size. The recording density of the HDS is generally dependent on the amount of information per data page, the size of the recording volume per data page, and the number of multiplexed holograms. In Fourier hologram configurations, use of multiple pixels per symbol does not reduce the recording density directly because a larger symbol size reduces both the focal spot size of the signal beam and the recording volume. However, the finite number of SLM pixels limits the number of symbols that can be recorded per data page and may also affect the data transfer rate. Therefore, it is important to develop methods that can realize continuous phase modulation that is suitable for the TIE method by using smaller numbers of SLM pixels from a data transfer rate viewpoint. In addition, the detection system for this continuous phase modulation requires use of multiple camera pixels per symbol, which may also adversely affect the transfer rate. Recent significant increases in the available numbers of camera pixels and transfer rates can mitigate this effect.

In recent years, many studies have been conducted with the aim of estimating object light phase using either a single diffracted light intensity image or multiple through-focus images as the input to neural networks [36–38]. In addition, in [39], a neural network was applied to digital holography to measure the accurate complex amplitude distribution from a single digital hologram. Although these techniques are quite attractive, current studies in the field [36–38] have only detected lightwave phases with relatively simple distributions, and the resolution and reproduction accuracies of the object images are also low. The methods described in [39] may be applicable to detection of the complex-amplitude-modulated signal beams used in HDS. However, the method proposed in this study offers advantages in terms of the vibration tolerance realized by non-interference-based measurements and use of simple and compact optical systems. Technically, in the future, it may be possible to classify/decode data pages in the complex-amplitude-modulated signal beams in HDS using a neural network in which one or more defocused images are used as the input to the network. However, in general, solving such difficult tasks will require training of more complex networks with more data for longer periods of time. Our method offers the advantage of obtaining relatively high-resolution phase distributions at high speed because phase recovery is achieved via the TIE method, which can use closed-form solutions. The task that is then imposed on our neural networks is the classification of symbol blocks that have been superimposed with noise. Because the spatial distribution of this noise has typical characteristics and supervised learning is possible, this task is relatively simple and is advantageous in terms of network size and training time requirements. Efforts to use neural network technology, including CNNs, in HDS decoders are very powerful and useful and will contribute to the realization of HDS systems.