1. Introduction

A three-dimensional (3D) display based on a holography reconstructs a wavefront emitted from a 3D scene. A wavefront reconstruction enables the display of 3D scene with all the depth cues of human vision. In addition, it is possible to reproduce a high-resolution image of a deep 3D scene contrary to the ray reconstruction such as multi-view or integral imaging method, in which the images located far from the display are blurred. However, as coherent light is used in the hologram recording and reconstructing process, speckle noise degrades the reconstructed image quality. In an electro-holographic display, since the hologram is calculated with a computer-generated hologram (CGH) technique, it is possible to reduce the speckle noise effectively by optimizing the CGH calculation method. In this paper, we propose a new calculation method for CGH that enables an effective speckle reduction and photorealistic reproduction.

Several approaches of speckle reduction have been proposed for electro-holographic displays. One of them uses reducing the temporal coherence of reconstruction light [1,2]. These approaches add blur to reconstructed image, therefore the advantage of holography, which is a reproduction of a deep 3D scene with high-resolution, could be lost. Another approach employs time averaging of several speckle patterns [3,4]. We call them as a “random-averaging method”. Although this approach doesn't add blur, this is not efficient because the effect of speckle contrast reduction is proportional to the square root of the number of multiplexed patterns. Recently, more effective methods have been proposed [5–7]. In particular, Takaki and Yokouchi has proposed a promising speckle reduction technique based on time-multiplexing of sparse point sources on the object surface by non-overlapped zone-plates [6].

Also, several CGH calculation methods have been investigated for electro-holographic displays. CGH calculation based on light-ray information, or the principle of holographic stereogram has been proposed [8–10]. Using light-ray information allows the reduction of calculation cost [8] and photorealistic reproduction of reconstructed image by utilizing advanced rendering techniques of conventional computer graphics (CG) [9]. This approach is called as a “ray-based method”. Recently, our research group proposed the CGH calculation method using ray-sampling plane (RS-plane). This method enables to realize high resolution image of a deep 3D scene in addition to the advantage of ray-based method [10], however, speckle reduction optimized for this method has not been considered yet.

In this paper, we present a method to apply the concept of the sparse point sources to the CGH calculated by using RS-plane. A simple model of speckle generation and the theory of speckle suppression in CGH are described in section 2. The motivation and the proposed method are described in section 3 and 4. The computer simulation results are described in section 5. Finally, discussions and summary are described in section 6 and 7.

2. Speckle generation and suppression in CGH

2.1 Speckle generation

We adopt a simple model of a speckle generation in a CGH reconstruction for estimating the speckle size in this paper. Figure 1 shows the setup of the CGH observation. The wavefront reconstructed by the CGH is propagated to observer's eye pupil. The lens of the observer's eye modulates the wavefront and forms the real image of the object on the retina.

 

Fig. 1 Model of speckle generation in CGH reconstruction.

Download Full Size | PDF

Assuming a coherent imaging system, the intensity distribution on the retina is written based on the Fresnel approximation as:

Speckle generation is originated from the interference of the adjacent object points. The image on the retina is simply expressed as the convolution of the wavefront on the focal plane and the PSF, which is originated from a low pass filtering functionality of the pupil, as shown in Eq. (1). Here, we consider an object as an aggregation of point sources, in which the point sources are separated less than Rayleigh resolution limit that is determined by the optical system of the eye. In the case the diffused object is imaged on the retinal surface, each point interferes with the adjacent point randomly since the points, which have random phases, are blurred by PSF convolution. In this way, speckle noise are generated on the reconstructed image of the CGH.

In addition to this, there are some sources of noise such as sampling, quantization, the recording and the reconstruction system. However it can be considered that the most influential reason of speckle generation is the interference between the object points on the observer's retina described above. In this paper, our purpose is to reduce this kind of speckle.

Here, we estimate the speckle size. If the object in Fig. 1 has a completely diffused surface, the average size of the speckle on the retina is estimated on the basis of the correlation width [12]. The average diameter of transverse direction is expressed as L_x,y, and axial direction is expressed as L_z. Then they are written as

In this paper, speckle noise is evaluated by speckle contrast C [12], which is written as:

2.2 Speckle suppression by sparse-points time-multiplexing

Let us now refine the approach for speckle suppression by a “sparse-points time-multiplexing” here considering the observation model shown in Fig. 1. In this method, an object surface consists of the sparse point sources and they are grouped into subsets as shown in Fig. 2(a).Then the image of each point on the retina is blurred by the PSF given in Eq. (2), but they are separated from adjacent ones by longer distance than the size of PSF main lobe to avoid the interference as shown in Fig. 2(b). Then, to fill the gap between the images of the sparse point sources incoherently, several CGH frames are displayed with higher speed than the temporal resolution of the human eye using a high-speed spatial light modulator (SLM). When the distance between adjacent points image after multiplexing is as small as the Rayleigh resolution limit [Fig. 2(c)], observer cannot perceive the point configuration of the object. In this way, the effective speckle reduction is realized without cutting down the spatial resolution. The main lobe width is given by Eq. (3) and that on the object surface becomes 2.44λ d_obj/D. Thus the minimum requirement is that the separation between points in the sparse subset must be several times larger than the main lobe width. As shown in Fig. 2(c), if the number of multiplexed point images is large enough, this method does not lower the spatial resolution of the reconstructed image.

 

Fig. 2 Concept of sparse-points time-multiplexing.

Download Full Size | PDF

This method works more efficiently than the random-averaging method. In the random-averaging method, the superposition of uncorrelated N speckle patterns can reduce the speckle contrast to $1/\sqrt{N}$. To decrease the number of the superposed speckle patterns, negatively-correlated speckle pattern is used [14]. Then, periodic dot patterns which are shifted smaller than the dot period each other have negative correlation. Therefore, if the separated distance of sparse point sources is several times of the PSF main lobe size, superposition of each frame can reduce the speckle contrast to less than $1/\sqrt{N}$ because of the negative correlation.

3. Motivation

We express the motivation for our proposal. As identified above, speckle suppression by the sparse-points time-multiplexing does not lower the spatial resolution of the reconstructed image and works more efficiently than the random-averaging method. On the other hand, we proposed the CGH calculation method using RS-plane (RS-plane method) [10]. One of the feature of this method is high resolution image of a deep 3D scene, however, the speckle noise degrades the image. Thus, in this paper, we adapt sparse-points time-multiplexing method to RS-plane method in order to suppress the speckle noise with high spatial resolution. Although, the method is not suitable for the ray-based CGH calculation techniques like RS-plane method, because it is a “point-based method”, in which objects are not consisted of light-rays but consisted of point sources. Therefore, we propose a novel speckle suppression method, in which sparse-points are realized by means of light-ray information processing on the RS-plane.

Here, we describe the comparison between the proposed method and the conventional methods [5–7]. A speckle suppression method using shift-averaging [5] is an effective approach, but it has been proposed for a holographic projection image and has not been applied for a 3D image. A CGH calculation using a zone plate modulation and the sparse point sources [6,7] realizes both shading reproduction and speckle suppression. However, it is not optimized for the PSF of human visual system that mainly causes speckle noise. Since the sparse-points are generated by the half zone plates arranged not to be overlapped each other, there are strong limitations as follows; (1) the size of the zone plate become significantly large if the reconstructed object is far from the hologram plane. Then it is almost impossible to arrange them not to be overlapped, and the method cannot be applied to the object located far from the hologram plane. (2) The number of the frames that should be multiplexed is large. In principle, the minimum requirement for the separation of the sparse-points can be considered as the PSF size determined by human visual system as described above. But by the use of non-overlapped zone plates, the separation becomes larger than the minimum requirement. These limitations are discussed again in section 6.

4. Speckle suppression in CGH calculation using RS-plane

4.1 CGH calculation using RS-plane

In this paper, the speckle suppression in the RS-plane method is discussed. RS-plane method is a ray-based method and is modified to avoid reduction of the spatial resolution in the image far from the hologram plane by introducing a ray-wavefront conversion [10]. The method is briefly reviewed in this section.

The concept of the method is shown in Fig. 3.In this method, at first the RS-plane is defined near the object location and the light-rays emitted from the object are sampled on the ray-sampling points on the RS-plane. In this step, light-ray information is sampled as projection images. Thus we can use conventional CG rendering techniques such as a ray tracing or an image-based rendering from a multi-view images of real or virtual object. Then each projection image is converted into a wavefront segment by Fourier transform likewise a holographic stereogram. Tiling these wavefront segments at the correspondent ray sampling points, the wavefront on the RS-plane is obtained. Next, the wavefront propagation from the RS-plane to the CGH plane and the interference of object and reference wave are simulated by using discrete Fresnel transform.

 

Fig. 3 CGH calculation using RS-plane.

Download Full Size | PDF

In this way, we obtain the CGH, which enables to reproduce a high-resolution image of a deep 3D scene. Using various CG techniques, it is also easy to calculate hidden-surface removal and occlusion processing. In the case of plural objects at different distance from CGH plane, this method is easily extended [10]. Applying to real scene with integral imaging and effective occlusion process are also studied [15,16].

Speckle noise appears on the reconstructed image in this method. The speckle noise is originated from the random phase of the ray-information added to equalize the energy of the wavefront segment. To date, speckle reduction technique optimized for this method has not been analyzed yet.

4.2 Outline of the proposed speckle suppression method

Introducing the concept of the sparse point sources to the RS-plane method, we can realize effective speckle suppression for the CGH that reproduces photorealistic and high-resolution deep 3D scene. In the RS-plane method, the wavefront is generated from the light-ray information without defining object points. Therefore, the sparse-points time-multiplexing method cannot be simply applied to the RS-plane method. The way to define the point sources on the object surface (object points) from the light-rays is described below.

In the proposed method, when the light-rays are sampled on the RS-plane, it is necessary to obtain the depth information of each light-ray. Then, we calculate the spatial coordinate from which the light-ray is emitted using the depth information. The coordinates obtained here corresponds the object points. We divide the object points into several subsets and the light-rays group corresponding to the subset are calculated by culling the light-rays corresponding to other subsets. Then, speckle suppression by sparse point sources are realized by light-ray culling process. After this, the optimal phases are added to the light-rays before converting to wavefront. The phase assigned to each light-ray is calculated to satisfy the condition that the phases of all light-rays from a certain object point on the object surface are matched. This phase adding technique is originated from phase-added stereogram (PAS) [17]. Recently, Kang et al. reported more study on the accurately compensated PAS (ACPAS) [18]. As shown in Fig. 4, the sparse object points are defined by culling light-rays to which phases are added. The CGH is separated to multiple frames that reconstruct different sets of sparse-points. Finally, the wavefront conversion, the propagation and the CGH cording are calculated for each frame similar to the RS-plane method. CGH frames are superposed in the reconstruction using the time-multiplexing.

 

Fig. 4 Concept of the proposed method.

Download Full Size | PDF

Since single object point is defined by a set of light-rays in the proposed method, the object point is able to have a various amplitude distribution in its angular spectrum similar to the modulated zone plate in [7]. This is obviously different from a pure point source, which has a uniform amplitude distribution in angular directions. We can call that object point as a “quasi-point source”. In contrast to a pure point source, which is expressed as a Dirac delta function, the quasi-point source has a finite width of complex amplitude distribution. Therefore, the proposed method enables not only to suppress speckle noise effectively, but also to realize various reflection properties of object surface, or easily process the hidden-surface removal or occlusion effect. But, if the width of quasi-point source is large, speckle suppression effect based on sparse-points multiplexing is degraded. Since the width of the quasi-point source is typically small, we assume the width is small enough compared with the interval of the sampling points.

In addition, if the depth information can be obtained, the proposed method is possible to apply not only a virtual scene but also a real scene captured by a multi-view camera or an integral imaging method.

4.3 Calculation procedure

In this subsection, the detailed calculation procedure is described. Here we consider the CGH calculation for a 3D scene with objects located in limited depth range. Figure 5 shows the setup of CGH calculation. Using RS-plane method, the distance between the objects and the CGH can be large.

 

Fig. 5 Setup for CGH calculation.

Download Full Size | PDF

Step 1: Obtain the light-ray and depth information

At First, RS-plane is defined near the objects. The shorter distance between objects and RS-plane is better for avoiding resolution reduction [10]. Also, it is possible to place the RS-plane in the middle of the object. Then the light-rays emitted from objects are sampled as projection images at sampling points on RS-plane as shown in Fig. 5. The pitches of the sampling points are ΔS_x and ΔS_y, and the pitches of the angular sampling are Δθ_x and Δθ_y. In this step, the depth information, which is the distance from RS-plane to the object surface, is also obtained for each light-ray. Intensity distribution of the light-rays is expressed as I_i,j[m,n] and depth distribution is expressed as d_i,j[m,n], where i and j are the indices of the ray-sampling point ($i\text{=0,}\pm \text{1,}\text{±2,}\text{…}$and$j\text{=0,}\pm \text{1,}\text{±2,}\text{…}$) and m and n are the pixel indices of projection image ($m\text{=0,}\pm \text{1,}\text{±2,}\text{…}$and$n\text{=0,}\pm \text{1,}\text{±2,}\text{…}$). $a_{i,j}[m,n]=\sqrt{I_{i,j}[m,n]}$ is the amplitude distribution.

Step 2: Define voxels in object space

A set of voxels is defined on Cartesian grid in the 3D space so as to cover the objects. Then, we assume that the light-ray is emitted from the center of voxels as explained in the step. 3. Object point is defined at the center of voxel. Undesirable artifacts are generated on the reconstructed image if there is unbalance of the number of light-rays emitted from the object points. These artifacts can be suppressed by matching the voxel pitch to the pitch of ray-sampling points ΔS_x and ΔS_y, and placing the center of voxel (the candidate of the object point) on the lines, which is perpendicular to the RS-plane and pass through the ray-sampling point shown in Fig. 6.The voxel pitch of axial direction is ΔS_z. The set of voxels is denoted as S_v hereafter.

 

Fig. 6 Defining voxel in the 3D space.

Download Full Size | PDF

Step 3: Assign light-rays to the voxels

Next, every light-ray is assigned to the corresponding voxel from which the light-ray is emitted as follows; first, using the depth information for each light-ray derived in the step. 1, the coordinates where each light-ray is emitted, $(x_{em},y_{em},z_{em})$ are calculated as shown in Fig. 7.They are expressed as

 

Fig. 7 Defining the object points from the light-ray emission coordinates.

Download Full Size | PDF

In this step, it is possible to be applied a back-project scheme, in which we find the intersection points of the light-ray with the object surface and then evaluate the amplitude and coordinate $(x_{em},y_{em},z_{em})$.

Step 4: Cull the light-rays

After the definition of the object points to all light-rays, the object points are culled to suppress the speckle generation as shown in Fig. 8.At first, the object points are divided into several groups, where the object points in each group are separated by a longer distance than PSF main lobe, which depend on optical system of the human vision and the observer’s distance from the object. In the time-multiplexing reconstruction method, each group of object points is associated with a single hologram and is related to the each frame. Therefore, when calculating the CGH corresponding to the one frame, the object point groups corresponding to the other frames are omitted. This process is similar to the method proposed in [6], but the division of the object point groups is calculated as the light-rays culling in the proposed method. Therefore, we substitute zero to the amplitude of the light-ray that is emitted from the culled object point in this process. Here, the voxel (object point) pitch ΔS_x and ΔS_y, are identical to the Rayleigh resolution limit of PSF, every other object point should be divided into different groups in order to avoid the interference of the PSF. Since better speckle suppression is realized by considering the interference of the first side lobe, we set the separation distance of object points to four times of object point pitch in a simulation of section 5.

 

Fig. 8 Light-ray culling instead of the object point culling to avoid interference of the PSF.

Download Full Size | PDF

Step 5: Add the optimal phase to the light-ray

Optimal phases calculated from the depth information obtained in step. 1 are added to the light-rays. In the CGH calculation using RS-plane, light-rays are regarded as a segmented plane wave. The segment size is equal to the distance of the ray-sampling points. We add the phases to the light-rays such that the phases of the light-rays (plane waves) emitted from a single object point are matched at the corresponding voxel center (object point). This calculation process is in the series of the research called phase-added stereogram (PAS) [17] or a coherent stereogram [18–20], which are studied to improve the resolution of the CGH of the ray-based method [17] or calculation speed of the CGH of the point-based method [18–20].

Let us now derive the phase that should be added to the light-rays using the concept of compensated phase added stereogram (CPAS) [19]. The wavefront of a segmented RS-plane emitted from the object point p, U_{CPAS_p}(ξ,η) is described as:

Step 6: Obtain the wavefront on the RS-plane

The amplitude of light-ray information is given by the square root of the projection image, as the pixel value of the image is considered to be the intensity. Adding phase by the CPAS theory, the set of light-rays that pass through a single ray-sampling point is converted into the small wavefront segment around the ray-sampling point by Fourier transform. Tiling the wavefront segment, the total wavefront on the RS-plane is obtained. The detail of the ray-wavefront conversion is described in [10]. In this step, using accurate phase added stereogram (APAS) technique, a quantization error of sampling light-rays is reduced. In APAS, the wavefront segment is cut out from the complex amplitude that is the transformed image of the oversampled light-ray information using FFT. The detail of the APAS is described in [20]. Consequently, we can use so called ACPAS [18], which is combination of APAS and CPAS.

Step 7: Simulate wavefront propagation from RS-plane to CGH-plane and encode CGH

The wavefront propagation from the RS-plane to the CGH-plane is calculated by discrete Fresnel transform. Several methods proposed for computing discrete Fresnel transform in digital holography [21,22]. Finally, we obtain the CGH by simulating the interference between the wavefront on the CGH plane and the reference wave.

4.4 Time-multiplexing reconstruction

In reconstruction step, multiple frames of CGHs are presented such that the reproduced images are incoherently superposed. In this paper, we consider time-multiplexing as in [6]. In the time-multiplexing reconstruction, the CGH pattern corresponding to each frame is transferred to a high-speed SLM. By displaying the several CGH frames in higher rate than the temporal resolution of human visual system, reconstructed images are multiplexed on the observer's retina incoherently. Since the ray-culling process suppresses the speckle noise of each frame, the speckle noise of the multiplexed image on the retina is also suppressed. In this method, spatial resolution of the CGH is not reduced.

5. Computer simulations

5.1 Example 1: planar diffused object

A computer simulation was carried out to confirm the effect of speckle reduction by the proposed method. At first, we simulated the case of a planar diffused object. Figure 9 shows the setup of the simulation and Table 1 shows the parameters of the simulations. An RS-plane was defined at 9.6mm in front of the planar diffused object of 6.4mm × 6.4mm square. In the simulation of reconstruction, the observer’s eye was placed at 300mm from the reconstruction image, where the eye pupil was 3.5mm in diameter and the distance between the eye pupil and the retina is 20mm. We assumed a CGH reconstructs the recording wavefront completely. And, the model of the human visual system on this simulation is same as the one described in subsection 2.1.

 

Fig. 9 Setup of the first simulation.

Download Full Size | PDF

Table 1. Parameters of the CGH Calculations

View Table

We used Open Graphics Library (OpenGL) [23] for modeling the object and obtaining light-ray and depth information. The interval of the object point of slanted 45 degree line was fitted to the Rayleigh resolution limit of the human visual system, which is L_x,y’/2 = 55.5μm, not to perceive the periodic dot pattern by the object points. And, the interval of the sparse object point was four times larger than that of horizontal and vertical direction, which is $55.5/\sqrt{2}\times 4=157$μm. Then, we divided the CGH into 16 frames for time-multiplexing reconstruction.

Figure 10 shows the simulation results of the reconstructed image on the retina. Figures 10(a) and 10(b) are the reconstructed image without speckle suppression and with the random-averaging method of 16 frames, respectively. In contrast to this, Figs. 10(d) and 10(e) are the single frame and time-multiplexed reconstruction image of the CGH calculated with the proposed method, where 16 frames are multiplexed. We verified the speckle suppression effect by estimating speckle contrast [Eq. (5)] of the simulation results shown in Fig. 10.

 

Fig. 10 Simulation results of the reconstructed image of planar diffused object CGH (a) without speckle suppression, (b) with the random-averaging method (16 frames), (d) with the proposed method (single frame) and (e) with the proposed method (16 frames). (c) and (f) are the horizontal cross section of the center of the (b) and (e).

Download Full Size | PDF

In the case of the random-averaging method, speckle contrast of the reconstructed image by 16 frames multiplexing shown in Fig. 10(b) is 0.24. It almost agrees with the theoretical value $1/\sqrt{16}=0.25$. On the other hand, speckle contrast by the proposed method is 0.15. This result is also expressed as the horizontal cross section of the center of Figs. 10(b) and 10(e) shown in Figs. 10(c) and 10(f), respectively. Therefore, we can say that the proposed method can suppress speckle contrast effectively. Remaining speckle noise is generated from the interference with the side lobes overlapped even though the image of point sources are separated as in Fig. 10(d). In order to realize the speckle contrast smaller than 0.05, it is needed to increase the frames for multiplexing with separating the interval of the sparse object points, apply other speckle suppression techniques such as the superposition of different polarization, or reduce the side lobes of the reconstructed point images.

5.2 Example 2: shaded 3D object

As a 3D object, a teapot modeled by OpenGL was used in the second example. The surface of the teapot object has angular dependent reflectance, which can be easily reproduced by ray-based rendering technique. At first, we obtained the light-ray and the depth information of the teapot using OpenGL. Then, the fringe pattern of the CGH was calculated with the proposed method. The parameters of the CGH is same as Table 1 and the setup of the simulation is shown in Fig. 11.

 

Fig. 11 Setup of the second simulation.

Download Full Size | PDF

The simulation results of the reconstructed image are shown in Fig. 12.Figure 12(a) is the reconstructed image by the hologram calculated without speckle suppression and Fig. 12(b) is the one calculated by the proposed method. The result shows that speckle noise can be effectively suppressed by the proposed method. Undesirable artifacts like lines are seen on Fig. 12(b) since the voxel pitch is unmatched to the pitch of ray-sampling points, but it is avoidable (see step. 2 of subsection 4.3). The experimental proof of this reconstruction method requires high-speed spatial light modulator, and it remains as a future issue.

 

Fig. 12 Simulation results of the reconstructed image of teapot CGH calculated (a) without speckle suppression and (b) with the proposed method (16 frames time-multiplexing).

Download Full Size | PDF

6. Discussions

Now we can indicate the novelty of the proposed method in comparison to the conventional method using non-overlapped zone-plates [6,7]. If the visible angle is same as this simulation ( ± 3.8deg as shown in Table 1) and the distance between the hologram and the object is 150mm, the zone-plate size becomes 20mm, which is larger than the hologram size. Therefore, it is impossible to display the object far from the hologram plane with same visible angle by using non-overlapped zone-plates. Moreover, in this simulation, we determined the interval of the sparse object points to consider the human visual system (157μm as shown in Table 1). On the other hand, it is depended on zone-plates size in [6,7], which means that the object far from the hologram plane requires more time-multiplexing frames independently of human visual system.

7. Summary

We proposed a new calculation method of CGH to suppress the speckle noise. In this method, CGH is calculated with the ray-information emitted from the object and speckle suppression is realized by culling the ray-information. We think that our approach makes a contribution to realizing high quality image of a holographic 3D display. Also, similar speckle suppression approach is proposed in the application of a lens-less holographic projection [13].

Merits of the proposed method are (1) more effective speckle noise reduction than random-averaging method, (2) easy to reproduce various light reflectance properties or process the occlusion by general CG techniques while it is difficult for the point source based method and (3) applicable not only virtual objects but also a real scene captured by multi-view camera or integral photography.

Future works are described below: (1) Evaluation of the speckle noise of the proposed method using the CGH which has more large size and large viewing angle, (2) effect of the spread of quasi-point source in speckle suppression should be considered as described in subsection 4.2, and (3) observer’s conditions including viewing position, focus position and pupil size, which affect the PSF are expected to be investigated.