1. Introduction

There has been growing interest in object recognition in two-dimensional (2D) and three-dimensional (3D) environments [1–10]. In digital holography, 3D information is recorded by optical interferometry. Recorded 3D information can be numerically processed to reconstruct holographic images at different longitudinal distances and perspectives. In the literature, various techniques using digital holography have been developed for 3D visualization, image encryption, and object recognition [3, 5–17]. A single wavelength is used in most of the researches. There has been a recognition research using color digital holography and the conventional correlation method [17].

In this paper, we address 3D color object visualization and statistical pattern recognition using multi-wavelength digital holography where multi-spectral 3D spatial information is recorded and reconstructed [12–15]. Holographic images are numerically reconstructed by the Fresnel transform method. The size of the reconstructed images can be controlled independently for perfect superimposition at different distances and wavelengths [14]. This technique is effective for the fast process such as real-time display and investigation including pattern recognition.

Statistical pattern recognition analyzes the multi-spectral information from color objects simultaneously. Two statistical pattern recognition approaches are adopted: principal component analysis (PCA) and mixture discriminant analysis (MDA). The PCA is a well-known dimensionality reduction technique to extract low dimensional features from high dimensional data [18–21]. The high dimensional vectors are projected onto a subspace spanned by eigenvectors of the covariance matrix of the training vectors. We extract low dimensional features which are characterized by different colors. Since the images are reconstructed by means of the superimposition reconstruction technique, no matching process is needed between multi-spectral information.

The multi-spectral features are simultaneously trained by the MDA. The multi-spectral features are complementary to one another for classification. The population distribution of one class is modeled by the weighted sum of the probability density functions of several components (clusters). In the Gaussian mixture model, each probability density function for the component is assumed to be Gaussian. Expectation maximization (EM) is a common approach to estimate the parameters of the Gaussian mixture model [19–27]. The MDA has been often adopted for cluster analysis, classification task, and density estimation [22–27]. The class-conditional probability density function for the population of each class is obtained using the PCA and the MDA during the training. For the decision making, the class of unlabeled images is determined by the maximum likelihood (ML) decision rule.

The organization of the paper is as follows. 3D object recording and reconstruction with multi-wavelength digital holography are described in Section 2. The PCA and MDA with the EM algorithm are described in Section 3. The ML decision rule and evaluation metrics are also discussed in Section 3. Experimental results are presented in Section 4. Conclusions follow in Section 5.

2. Multi-wavelength digital holography

The use of multi-wavelength digital holography has been researched for 3D color display [12–15]. Figure 1 illustrates the optical setup for multi-wavelength digital holography. In our approach, two lasers (l ₁ and l ₂) with different wavelengths are used. One is in the red region (λ₁=632.8 nm), and the other is in the green region (λ₂=532.0 nm). The optical configuration is arranged to allow the two lasers to propagate along the same paths either for the reference or the object beams. The reflecting prism which is in the path of the red laser beam permits the matching of the optical paths of the two interfering beams inside the optical coherent length of the laser. The object (toy warrior) as shown in Fig. 1 is placed at a distance 500 mm from the CCD (charge-coupled detector) array. Two holograms are recorded with two wavelengths. The holograms are reconstructed separately by the Fresnel transformation method [16].

It is noted that different details are visible in reconstructed images since different color information is recorded according to the wavelengths. Moreover, the sizes of reconstructed images are different since the reconstruction pixel size depends on the wavelength. Reconstruction is obtained through the pixel size (Δx, Δy) of the CCD array which is different from the pixel size (Δx′,Δy′) in the image plane, where Δx and Δy are the resolutions of the CCD in the x and y directions, respectively, and Δx′ and Δy′ are the resolutions of the image plane in the x and y directions, respectively. They are related by Δx′=d_sλ/N_x Δx and Δy′=d_sλ/N_yΔy, where d_s is a longitudinal depth, λ is the wavelength, and N_x and N_y are the number of CCD pixels in the x and y directions, respectively.

Recently, it has been demonstrated that the size of the reconstructed images can be controlled independently for perfect superimposition between the reconstructed images [14]. For the perfect superimposition, we enlarge the number of pixels by zero padding. Let us say that N _x1 is an increased size of the hologram plane by zero-padding in the x direction. If one hologram has been recorded with wavelength λ₁, the other one with λ₂, and λ₁>λ₂, then, the number of pixels of the hologram of the wavelength λ₁ is changed to

where N _x2 is the size of the hologram plane in the x direction with the wavelength λ ₂. Consequently, we obtain the same resolution for reconstructed images for the holograms of different wavelengths:

where Δx ₁′ and Δx ₂′ are the resolutions of the image plane in the x direction with wavelength λ ₁ and λ ₂, respectively. The reconstructed image size in the y direction is controlled in the same way. In the experiments, the size of the hologram plane (N _x2) is 1024 pixels, therefore, N _x1 becomes 1218 pixels.

 

Fig. 1. Optical set-up for multi-wavelength digital holography using two lasers, M: mirror, BS: beam splitter, BE: beam-expander, RP: reflecting prism, CCD: charge coupled device.

Download Full Size | PDF

3. Statistical pattern recognition techniques

In this section, first, the PCA is presented, and then the MDA with the EM algorithm is discussed. Finally, the ML decision rule and evaluation metrics are presented.

3.1 Principal component analysis

Let a column vector composed of the pixel values of the reconstructed images be one realization of a random vector x∈R ^d×1, where R ^d×1 is d-dimensional Euclidean space, and d is the same with the number of pixels in the reconstructed image. The PCA projects the d-dimensional vectors onto the l dimension subspace (l≤d) [18–21]. For a real d-dimensional random vector x, let the mean vector be µ_x=E(x), and the covariance matrix be Σ_xx=E(x-µ_x)(x-µ_x)^t, where the superscript t denotes transpose. The space for the PCA is spanned by the orthonormal eigenvectors of the covariance matrix, that is, Σ_xxE=EV where the column vectors of E are normalized eigenvectors e _i’s, i.e. E=[e ₁,…,e _d], and the diagonal matrix V is composed of eigenvalues v_i’s, i.e. V=diag(v ₁,…, v_d). For the PCA, the projection matrix W_p is the same as the eigenvector matrix E. Therefore, a projected vector y by W_p is

The PCA diagonalizes the covariance matrix of y, i.e. Σ_yy=E(y-µ_y)(y-µ_y)^t=V where µ_y=E(y). If we choose the projection matrix W_p=[e ₁,…,e _l], the subspace is spanned by the corresponding l eigenvectors. It is a well known property of the PCA that by choosing l eigenvectors of the largest eigenvalues, the mean-squared error between a vector x and a restored vector x̂ is minimized. The mean-squared error is defined as

and the restored vector x̂ is defined as

where v_i’s are eigenvalues of v_d≤v _d-1,…, v ₂≤v ₁. The PCA can reduce the dimension of the vectors while retaining dominant features of the object structure and reducing redundant and noisy data.

3.2 Mixture discriminant analysis

To deal with different visible features, the population distribution of the color holographic images is modeled as the mixture of several component probability density functions [22–27]. If the class j is composed of G_j components of the probability density functions, the class-conditional probability density function is

where the vector y is a projected vector onto the subspace by the PCA, w_jk denotes an event that the vector y belongs to the component k of the class j, N_c is the number of classes under investigation, P(w_jk) is the probability that the event w_jk occurs.

In the Gaussian mixture model, the probability density function of each component is assumed to be multivariate Gaussian as

where N(·) denotes the multivariate Gaussian distribution, and µ _jk and Σ_jk are the mean vector and the covariance matrix of the component k in the class j, respectively. Therefore, solving the MDA with the Gaussian mixture model is equivalent to estimating three unknown parameters (P(w_jk), µ _jk, Σ_jk) for each Gaussian component of the classes.

Let the log-likelihood function of the joint density with n_j training images of the class j be

where $p_j({\mathbf{y}}_{j1},\dots ,{\mathbf{y}}_{{\mathrm{jn}}_j})$ is the joint probability density function of n_j observations from the class j. The maximum likelihood solution of Eq. (8) is obtained as [19–21]

where _jk µ̂, Σ̂_jk, and P̂(w_jk) are the estimators for the mean, covariance and mixing weight, respectively.

As shown in Eqs. (9)–(12), the maximum likelihood solution is a set of highly non-linear coupled equations. The EM algorithm is a popular approach to solve them. The EM algorithm uses a set of training data to iteratively estimate the parameters until convergence. Figure 2 shows the block diagram of the EM algorithm for the class j; i represents the number of the iteration, and i _max and ε are the termination criteria for the iteration. The EM procedure is as follows: for initialization, we set ${\widehat{\mathbf{\mu }}}_{jk}^0,{\widehat{\Sigma }}_{jk}^0,{\widehat{P}}^0(w_{jk})$ for k=1,…,G_j; during E step, we compute P;̂(w_jk|y _jt) for k=1,…,G_j, and t=1,…,n_j as in Eq. (9); and during M step, we compute µ;̂_jk, Σ̂_jk, P;̂(w_jk) for k=1,…,G_j as in Eqs. (10)–(12). The E and M steps are iterated until $\mid \frac{L^i-L^{i-1}}{L^{i-1}}\mid \le \epsilon$ or i>i _max. In the experiment, we set ε at 10^-50 and i _max at 100. These E-step and M-step are repeated for j=1,…, N_c.

 

Fig. 2. EM algorithm

Download Full Size | PDF

The dimension of the vector y on the PCA subspace should be pre-determined for the PCA. In the experiments, several dimensions for y are tested. The number of clusters G_j in Eq. (6) is another factor to be considered carefully. We chose the cluster number to be the same as the number of wavelengths which is two in the experiments. The initialization of the parameters is the other important factor. We use the Linde-Buzo-Gray initialization satisfying the Lloyd’s optimality conditions [28,29]. In our experiments, the performance of Linde-Buzo-Gray initialization has shown to be better than initialization using typical k-means clustering [18].

3.3 Maximum likelihood decision rule

Our decision rule compares the class-conditional probability density functions with unknown test image. Let a test vector y _test be the unlabeled vector on the PCA subspace as in Eq. (3). In the experiments, a test vector corresponds to a reconstructed image from one hologram. Therefore, a test vector y _test contains a single spectral feature of the object.

The class-conditional density functions determines the class of the test vector as

where C_ĵ is the set of the class ĵ, and p̂_j(y) is the class-conditional probability density function with the estimated parameters.

To evaluate the performance, we calculate two performance measures: correct classification rate and false classification rate which are, respectively defined as

4. Experiments and simulation results

Color information of three objects (toy warriors) is recorded with two different wavelengths as illustrated in Fig. 1. The height of the objects is around 12 mm. They are placed at a distance of 500 mm from the CCD array since it is the minimum distance to obtain the whole object reconstruction image. The power levels of λ₁ and λ₂ are 35 mW and 50 mW, respectively, and the sizes of the beam diameter for λ₁ and λ₂ are 1.2 mm and 1.8 mm, respectively. We control the reflecting prism (RP) to equalize the paths of the object beam and the reference beam, in such a way that the optical path difference is inside the coherence length of the laser. It is noted that the coherence length of the red laser is about 20 cm, while the green is in meters. The CCD array is JAI Mod. CV M4. The spatial resolution of the CCD is 1280×1024 pixels and the size of each pixel is 6.7 µm.

Holographic images are computationally reconstructed at various image planes of different depths. The perfect superimposition technique in [14] is used to obtain holographic images of three objects. One hundred images are reconstructed at every millimeter from 451.0 mm to 550.0 mm for each hologram. Figures 3–5 show the movies of reconstructed images for three objects captured by two wavelengths. Different visible features appear in the same object images according to the wavelengths. Only 50 images are shown in each movie which are reconstructed at 451, 453, 549 mm. In the figures, the holographic images are cropped to be 500×400 pixels assuming that the images are segmented. We present the color (RGB) images in Fig. 6. The objects are reconstructed at 500 mm. The red component is identical with the holographic images using the red laser (λ₁) and the green component is the green laser (λ₂).

 

Fig. 3. Movies of 50 reconstructed images from the class 1, (a) (the movie file size: 2.10 MB) λ=632.8 nm [Media 1], (b) (the movie file size: 2.10 MB) λ=532.0 nm [Media 2].

Download Full Size | PDF

 

Fig. 4. Movies of 50 reconstructed images from the class 2, (a) (the movie file size: 2.10 MB) λ=632.8 nm [Media 3], (b) (the movie file size: 2.10 MB) λ=532.0 nm [Media 4].

Download Full Size | PDF

 

Fig. 5. Movies of 50 reconstructed images from the class 3, (a) (the movie file size: 2.10 MB) λ=632.8 nm [Media 5], (b) (the movie file size: 2.10 MB) λ=532.0 nm [Media 6].

Download Full Size | PDF

 

Fig. 6. Color holographic images, (a) object 1, (b) object 2, (c) object 3.

Download Full Size | PDF

For training, two images are randomly chosen from a set of 100 holographic images, therefore, the total number of training images is four for each class. We classify other 196 images which are not trained. The same experiment is repeated 100 runs with differently selected training images. The correct classification and false classification rates in Eqs. (14) and (15) are averaged over 100 runs. It is noted that we train the multi-spectral information of objects but classify unknown images with a single visible feature. Although the multi-spectral information is embedded into the system, we can recognize the objects with the single spectral feature requiring less resource for recording and reconstruction.

The same experiments are performed with 6, 10, and 20 training images from one class. Figure 7 shows the average correct and false classification rates when the dimension of the PCA subspace is one. With the increased number of training data higher correct classification rates and lower false classification rates are produced since the class-conditional probability density functions are estimated more accurately. Figure 8 illustrates the results when the dimension of the PCA subspace is two.

 

Fig. 7. Classification results when the dimension of y is one, (a) averaged correct classification rates over 100 runs, (b) averaged false classification rates over 100 runs.

Download Full Size | PDF

 

Fig. 8. Classification results when the dimension of y is two, (a) averaged correct classification rates over 100 runs, (b) averaged false classification rates over 100 runs.

Download Full Size | PDF

It is noted that there is no image processing techniques applied to the reconstructed images for noise cancellation although the noise is caused by the DC term and coherent light diffraction. We may expect better visualization and recognition results with any conventional noise reduction technique on the images.

5. Conclusions

In this paper, the multi-spectral information of 3D objects is recorded and reconstructed with multi-wavelength digital holography. The same size images are reconstructed using perfect superimposition technique. The multi-wavelength digital holography provides spatial and spectral information of 3D objects. The statistical pattern recognition techniques handle the color information of objects simultaneously. The PCA extract low dimensional feature vectors from the reconstructed images and the MDA trains the multiple features to estimate the class-conditional density function. The proposed system is proven to provide the discrimination capability for different color objects with a few training images.