1. INTRODUCTION

Three-dimensional displays based on computer holography are a promising technology in the pursuit of ultimate digital displays. However, a gigantic space-band product (SBP) is necessary for reconstructing three-dimensional (3D) images with a given size and viewing angle in computer holography. This makes it difficult to realize holographic displays in practical applications.

In many cases, a computer-generated hologram (CGH) reconstructs nonphysical virtual objects from their numerical models. This is similar to processes used in computer graphics (CG). Therefore, computer holography and CG require realistic rendering techniques, such as the rendering of specular objects, in order to reconstruct images that appear as real as possible.

Actual rendering techniques in computer holography inherently depend on a way of generating object fields from the numerical model. Techniques for the numerical synthesis of object fields can be classified into three large groups. One group is techniques based on multi-viewpoint images (MVI) [1–3]. In this case, the base object is many two-dimensional (2D) images rendered by CG. Thus, rendering techniques in CG directly provide this in computer holography. This is an advantage of MVI-based techniques. However, MVI-based techniques produce only a holographic stereogram. Thus, they cannot completely solve the problem of vergence–accommodation conflicts.

The second group comprises the most popular technique for object field synthesis, which is the point-based method [4,5]. In this technique, a cloud of point sources of light forms an object. This technique is sometimes simply referred to as the point cloud. It is a very simple but hugely time-consuming technique, because an enormous number of point sources are required to form an object. Efforts to reduce the required computation time continue to receive much research attention [6–10]. In the point cloud technique, reconstructed objects commonly have diffuse surfaces. A few studies have proposed methods for rendering specular objects [11,12]. But these methods have never created any large SBP CGHs, because of the long computation time required.

The third group for the numerical synthesis of object fields is the polygon-based method [13–25]. Instead of point sources, polygonal surface sources are used to form an object [13]. The number of surface sources is much smaller than the number of point sources in a point cloud. This makes the polygon-based method well suited for creating large SBP CGHs. In practice, large-scale CGHs composed of more than a billion pixels have been produced using the polygon-based method [16,20–26]. These are referred to as high-definition CGHs.

The original technique for the polygon-based method creates CGHs that reconstruct diffuse objects with flat shading [13]. The polygon-based method is similar to techniques in CG. This means that smooth shading techniques in CG, such as Gouraud and Phong shading, can easily be applied to polygon-based CGHs [21]. In practice, high-definition CGHs that reconstruct diffuse curved objects have been created using these techniques.

Several techniques have been proposed for reconstructing specular surfaces in the polygon-based method [18–20]. Phase modulation based on a microfacet model of metallic surfaces has been proposed for reconstructing specular objects [18]. Reproducing the specular reflection of real metallic materials has also been attempted, using an atomic force microscope and the finite-difference time-domain method [19]. Unfortunately, the validity of these techniques has not been confirmed in large SBP CGHs. We also proposed a spectrum control technique, which was based on the Phong reflection model. We verified that the computation time is sufficiently short to allow it to be applied to high-definition CGHs having a large SBP [20].

The above-mentioned techniques have only been proposed for reconstructing flat surfaces. We have also proposed a technique for specular smooth shading using fragmentary plane waves to change the direction of light emitted from a polygon surface [22–24]. This technique is based on spectrum control, proposed in Ref. [20]. In the current paper, we refine these techniques and integrate them with the Phong reflection model, which is a well-known rendering technique in CG. Furthermore, an important parameter of the proposed technique is discussed for proper rendering. An actual high-definition CGH is created using the parameter to verify the validity of the proposed technique. The measured computation time shows that the proposed technique is sufficiently practical for creating large SBP CGHs.

Conventional rendering techniques in the polygon-based method are explained in Section 2, for readers’ convenience. The proposed rendering technique of specular curved surfaces is described in Section 3, and verified by creating an actual high-definition CGH in Section 4. We conclude our findings in Section 5.

2. RENDERING OF DIFFUSE AND SPECULAR SURFACES IN POLYGON-BASED CGHs

In this section, we summarize the principle of the polygon-based method [13,16] and its rendering techniques [20,21].

A. Principle of the Polygon-Based Method

1. Three Coordinate Systems

In the polygon-based method, three different coordinate systems are used for rendering, as shown in Fig. 1. Hologram coordinates are represented by $(\widehat{X},\widehat{Y},\widehat{Z})$. The hologram is placed in the $(\widehat{X},\widehat{Y},0)$ plane, and the objects are generally arranged in the space where $\widehat{z}<0$. Parallel local coordinates are defined for each polygon, and are given by $({\widehat{x}}_n,{\widehat{y}}_n,{\widehat{z}}_n)$ for a polygon $n$. These coordinates are similar to the hologram coordinates, but the origin is placed in the polygon surface. Tilted local coordinates $(x_n,y_n,z_n)$ are also defined for each polygon, and share the origin with the parallel local coordinates. However, these coordinates are tilted so that the surface of the polygon is placed in the $(x_n,y_n,0)$ plane. The identifier $n$ is omitted in the following discussion, to concentrate on the rendering of a given polygon, and to simplify the notation.

 

Fig. 1. Three coordinate systems used in the polygon-based method.

Download Full Size | PDF

Two local coordinates can be converted to each other using a rotation matrix $\mathbf{T}$. For example, the position vector $\widehat{\mathbf{r}}={[\begin{array}{ccc}\widehat{x}& \widehat{y}& \widehat{z}\end{array}]}^{\mathrm{T}}$ in the parallel local coordinates can be converted to the position vector $\mathbf{r}={[\begin{array}{ccc}x& y& z\end{array}]}^{\mathrm{T}}$ in the tilted local coordinates by

2. Surface Function

A polygon is represented by the complex function $h(x,y)$ given in the $(x,y,0)$ plane of the tilted local coordinates. This complex function is referred to as a surface function, and is a distribution of the complex amplitude. This function represents the wave field of light emitted by the polygon [13]. The polygons are often not parallel to the hologram, so several steps are necessary to obtain the wave field in the $(\widehat{x},\widehat{y},0)$ plane of the parallel coordinates. In this procedure, the surface function is first subjected to a Fourier transformation as follows:

 

Fig. 2. Diffuse reflection in a (a) broadband field, and in narrowband fields, (b) without, and (c) with spectrum remapping.

Download Full Size | PDF

3. Spectrum Remapping

If the bandwidth of $H(u,v)$ is sufficiently large, then light emitted by the polygon always reaches the hologram, as shown in Fig. 2(a). In contrast, the spectrum $H(u,v)$ of limited bandwidth may not deliver the light to the hologram, as shown in 2(b). However, a wider bandwidth requires more computational effort to achieve the numerical calculation. To change the direction of the limited bandwidth light toward the hologram, as shown in 2(c), the spectrum $H(u,v)$ must be shifted as follows:

4. Coordinate Rotation in Fourier Space

The surface function spectrum $\widehat{H}(\widehat{u},\widehat{v})$ in the parallel coordinates is obtained from $H^{\prime }(u,v)$, using coordinate rotation as follows [13]:

B. Surface Function Based on the Phong Reflection Model

1. Principle of the Phong Reflection Model

The Phong reflection model is a well-known reflection model used for rendering specular surfaces in CG [27]. Figure 3 schematically shows the Phong reflection model for a given polygon, whose normal vector is $\widehat{\mathbf{N}}$. Here, the components of vectors with and without hat symbols are given in the parallel and tilted coordinates, respectively. Since the normal vector in the tilted coordinates is $\mathbf{N}={[\begin{array}{ccc}0& 0& 1\end{array}]}^{\mathrm{T}}$ according to the coordinate definition, then the normal vector is given by $\widehat{\mathbf{N}}={\mathbf{T}}^{-1}\mathbf{N}={[\begin{array}{ccc}{a}_7& a_8& a_9\end{array}]}^{\mathrm{T}}$, where ${\mathbf{T}}^{-1}$ stands for an inverse matrix of $\mathbf{T}$. Note that a rotation matrix satisfies ${\mathbf{T}}^{-1}={\mathbf{T}}^{\mathrm{T}}$ in general.

 

Fig. 3. Schematic illustration of the Phong reflection model.

Download Full Size | PDF

Supposing that illuminating light traveling along the unit vector $\mathbf{L}$ irradiates the polygon, then the brightness of the polygon in the Phong model is given by

The Phong model classifies reflected light into three types. In the right-hand side of Eq. (10), the first term represents ambient light whose direction is independent of both the direction of illumination and the normal of the polygon. In contrast, the second and third terms are dependent on these two factors. The second term corresponds to diffuse reflection whose light also spreads over all directions. However, the surface brightness depends on the incident angle $\theta$ of the illuminating light. The third term corresponds to specular reflection, whose light spreads over a narrow region around the regular reflection. Here, the parameter $\alpha$ is a shininess constant that determines the degree of diffuseness, i.e., a bigger $\alpha$ results in a smaller diffusiveness. The three coefficients $K_a$, $K_d$, and $K_s$ are the weights of ambient light, diffuse reflection, and specular reflection, respectively.

Because the Phong model is a simple rendering model, it can be readily applied to the polygon-based method. The surface function based on the Phong model is written as follows:

The diffuse and specular surface functions have forms of

2. Spectral Envelope of the Surface Function

The spectrum of the surface function in Eq. (13) is written as follows:

 

Fig. 4. Schematic illustration of the spectral envelopes of the diffuse and specular reflections. $\theta$ is the angle of regular reflection shown in Fig. 3 and $\lambda$ is a wavelength.

Download Full Size | PDF

We previously proposed a technique to produce the surface function of a specular surface. The spectrum $H_2(u,v;\mathbf{R})$ in Eq. (17) of Ref. [20] can be used as the specular spectrum; that is,

3. Cancelation of Spectrum Remapping

When the spectrum remapping described in Section 2.A.3 is used, additional compensation is required in $H_s(u,v;\mathbf{R})$. This is because spectrum remapping changes the direction of light emitted from the polygon. This is problematic for specular reflection where the direction of the reflected light is designated by $\mathbf{R}$. To cancel the shift by spectrum remapping and eliminate its unwanted effect, spectrum $H_2(u,v;\mathbf{R})$ must be shifted in advance as follows:

3. RENDERING OF SPECULAR CURVED SURFACES

In curved surfaces, the normal vector smoothly changes depending on the surface position, as shown in Fig. 5(b). Thus, when illuminating light is reflected by the curved surface, the direction of regular reflection from the surface smoothly changes. In contrast, light reflected by a planar polygon has the same direction anywhere in that polygon surface, as in Fig. 5(a). This gives flat specular shading, and causes reconstruction of angled facets. Reducing the size of individual polygons and increasing their number can make the surface look smoother. However, increasing the number of polygons usually increases the required computational effort and calculation time.

 

Fig. 5. Reflection from (a) planar polygons and (b) a curved surface.

Download Full Size | PDF

A. Principle of Smooth Shading Based on Phong Shading

Phong shading is a well-known technique for specular smooth shading in CG [27]. Here, note that the Phong shading technique is different from the Phong reflection model mentioned in Section 2.

Figure 6(a) schematically shows the normal vector $\mathbf{N}$ and reflection vector $\mathbf{R}$ in a curved surface. In the Phong shading technique, to imitate a curved surface, the normal vector of a polygon is not constant in the polygon surface, but varies depending on the position, as shown in Fig. 6(b). In practice, normal vectors ${\mathbf{N}}_m(m=0,1,2,\dots )$ are obtained using interpolation between the vertex normal vectors of the polygon, which are given by the average of the normal vectors of neighboring polygons. The surface brightness corresponding to each normal vector ${\mathbf{N}}_m$ is determined using Eqs. (10) and (11). As a result, if the normal vectors are interpolated sufficiently densely, then the change in brightness is sufficiently smooth that the polygon appears to be a curved specular surface in CG rendering.

 

Fig. 6. Schematic illustrations of (a) normal vectors of a curved surface, (b) interpolated normal vectors in the Phong shading technique, and (c) field emission in the technique proposed in the current study.

Download Full Size | PDF

The original Phong shading technique in CG does not work well in holography. This is because it changes only the surface brightness, and does not change the direction of light emitted by the polygon surface. Let us emphasize that viewers of a hologram can change their viewpoints, unlike in CG. Thus, the appearance of the polygon surface should smoothly change as the viewpoint moves in computer holography. To imitate a curved surface, each portion of a polygon surface must emit light in different directions represented by ${\mathbf{R}}_m$, as shown in Fig. 6(c). Here, the reflection vector ${\mathbf{R}}_m$ is given by Eq. (11) and the interpolated normal vectors ${\mathbf{N}}_m$.

B. Specular Surface Function for Smooth Shading

To change the field direction for each portion of a polygon, we divide the surface function into rectangular segments corresponding to the reflection vector ${\mathbf{R}}_m$. We then change the direction of the light by multiplying each segment of the surface function by a plane wave traveling in the direction of ${\mathbf{R}}_m$. The fragmentary plane wave limited inside the rectangle of segment $m$ is given by

 

Fig. 7. Example of fragmentary plane waves. Each fragment corresponds to a segment of the specular surface function.

Download Full Size | PDF

In ambient light and diffuse reflection, light emitted from a surface inherently spreads over all directions, as mentioned in Section 2.B. Therefore, the direction of light does not need to be changed in these fields. The original Phong shading technique in CG, i.e., change in brightness of the surface, achieves smooth shading with regard to ambient and diffuse components even in computer holography [21]. Only specular reflection requires the fragmentary plane waves to render a curved surface.

A surface function for specular flat shading is given by the inverse Fourier transformation of Eqs. (17) or (18) as follows:

C. Procedure for the Rendering of Specular Curved Surfaces

Figure 8 shows the procedure for generating the surface function of a specular curved surface. The diffuse surface function shown in (b) gives ambient light and diffuse reflection. The specular surface function shown in (a) has a narrowband spectrum, whose width is specified by a shininess constant $\alpha$, and whose peak is at the origin.

 

Fig. 8. Procedure for rendering a specular curved polygon. (a) Specular surface function for flat shading, (b) diffuse surface function, (c) fragmentary plane waves, and (d) spectrum of the surface function for the specular curved surface.

Download Full Size | PDF

To change the direction of light inside the polygon surface, the field (a) is multiplied by the fragmentary plane waves shown in (c). The direction of the plane waves is determined by Eq. (11), using the illumination vector and interpolated normal vectors based on the Phong shading technique. The final surface function is given by the weighted sum of the diffuse and specular surface functions using the weights $K_a$, $K_d$, and $K_s$. Fig. 8(d) shows the spectrum of the final surface function.

4. CREATION OF A HIGH-DEFINITION CGH

A high-definition CGH composed of more than four billion pixels was actually created for verifying the validity of the proposed technique.

A. Segment Size

It is important to choose an appropriate segment size in specular surface functions. The field direction is controlled using fragmentary plane waves in the proposed method. If the segment size is too small, the fragmentary plane waves may not be able to properly control the field direction. In contrast, a segment size that is too large will most likely result in a decrease in the smoothness of the curved surface.

We calculated some object fields with different segment sizes to investigate the influence of segment size on the reconstruction of curved surfaces. The object model we used in this experiment is called The Venus. This model is composed of 718 front-face polygons. To make clear the effect of segment size, the object field was calculated by specular reflection only, i.e., $K_a=K_d=0$. The wavelength is 633 nm, and the sampling interval of the surface function is 320 nm. The object field was calculated with $16\mathrm{K}\times 16\mathrm{K}$ ($1\mathrm{K}=1024$) samplings and 1.0 μm sampling intervals for the object having a height of 1.4 cm. The reconstruction is simulated by numerical image formation using virtual optics [26].

Figure 9 shows the simulated reconstruction. Here, the sampled surface function is divided into $M\times M$ sampling square segments. It is verified that the curved surface is degraded for segment sizes of less than $M=8$. The reconstruction is almost unchanged for segment sizes of $M\ge 16$. Thus, we set out to create an actual high-definition CGH with $M=32$.

 

Fig. 9. Simulated reconstruction of object fields calculated using different segment sizes.

Download Full Size | PDF

B. Calculation of High-Definition CGH

A high-definition CGH named “The Metal Venus II” was created for verifying the proposed method. Figure 10 shows the 3D scene. The object of the Venus is the same model as that of The Metal Venus I created in our previous work, in which the specular flat shading technique was proposed [20]. The object of the Venus has a height of 5.7 cm. The parameters used for generating the CGH are summarized in Table 1. The sampling interval of the surface functions is less than a half of a wavelength, to avoid aliasing errors of the fragmentary plane waves. Thus, spectrum remapping is not used in this case.

 

Fig. 10. Three-dimensional scene of The Metal Venus II.

Download Full Size | PDF

Table 1. Parameters Used for Creating The Metal Venus II

View Table

Computation was executed using a PC with two Xeon X5680 (3.33 GHz) CPUs and 144 Gbytes of memory. The total number of real cores is 12. Figure 11 shows the computation time of The Metal Venus II (this work) and The Metal Venus I [20] for comparison.

 

Fig. 11. Measured computation time of The Metal Venus I [20] and The Metal Venus II (this work).

Download Full Size | PDF

These CGHs show almost the same 3D scene, which is very similar to “The Venus” CGH reconstructing diffuse surfaces [16]. Calculation of the object field in the hologram plane is also very similar to The Venus and mainly divided into three steps. The field of the wallpaper is first generated and propagated to a plane crossing the Venus object for occlusion processing by the silhouette method [25]. The computation time for this step is indicated as “1st propagation” in Fig. 11. The Venus field is then calculated using the proposed technique and added to the background field. This field is finally propagated to the hologram plane. This is the “2nd propagation” in Fig. 11.

The computation time is mainly consumed in the two propagations and in computing the Venus field. The total computation time of The Metal Venus II was 3.7 h, which was a few minutes longer than that in The Metal Venus I. This result verifies that specular curved objects can be calculated as fast as specular flat objects, using the proposed method.

C. Optical Reconstruction of the High-Definition CGH

The calculated object field was numerically interfered by a reference wave to generate the fringe pattern of the CGH. The fringe image was finally binarized by a threshold, and printed using laser lithography [16,26]. The optical reconstruction of The Metal Venus II is shown in Fig. 12(b). Here, a He–Ne laser is used for the illumination light source. Optical reconstruction of The Metal Venus I, rendered by specular flat shading, is also shown in Fig. 12(a) for comparison. These pictures clearly show the difference between specular flat and smooth shadings.

 

Fig. 12. Optical reconstruction of (a) The Metal Venus I having specular flat surfaces [20] and (b) The Metal Venus II created using the proposed method (see Visualization 1). A He–Ne laser is used for the illumination light source.

Download Full Size | PDF

Figure 13 and Visualization 2 show the optical reconstruction of The Metal Venus II using a red LED. Continuous motion parallax as well as specular curved surfaces are verified in these pictures.

 

Fig. 13. Optical reconstruction of The Venus II by an ordinary red LED. The pictures are taken from different angles (see Visualization 2).

Download Full Size | PDF

5. CONCLUSION

We propose a new realistic rendering method to create high-definition CGHs of specular curved objects. The proposed method is based on specular flat shading proposed in our previous work, and the Phong shading technique in CG. In the proposed method, a surface function is divided into small rectangular segments, in order to change the direction of the reflected light.

A high-definition CGH composed of four billion pixels was calculated using the proposed method. The measured computation time is comparable to that for specular flat shading, and is therefore sufficiently short to allow high-definition CGHs to be created. Optical reconstruction of the fabricated CGH verifies that the proposed method can reconstruct specular curved objects.