Efficient texture mapping by adaptive mesh division in mesh-based computer generated hologram

Yeong-Min Ji; Hanju- Yeom; Jae-Hyeung Park

doi:10.1364/OE.24.028154

1. Introduction

Texture mapping is a useful technique in computer graphics that makes it possible to create lifelike and realistic three-dimensional (3D) objects at a small computational cost. In computer graphics, a 3D object is generally composed of numerous triangles. Each triangle is filled with a uniform color considering material color and lighting effect. As the number of the triangles increases, more details of the object can be rendered. At the same time, however, the computational load also increases with the increased number of the triangles, requiring more computation time. In the texture mapping method, the area inside each triangle is filled not with a uniform color but with corresponding two-dimensional (2D) texture image data which is pre-calculated. Therefore, the detailed representation of the object is possible without increasing the number of the triangles. This texture mapping method has been widely used in computer graphics [1–3].

Recently, some techniques that apply the texture mapping to triangular-mesh-based computer generated hologram (CGH) have been proposed. The triangular-mesh-based CGH methods synthesize the hologram of 3D objects by accumulating the optical field of individual triangular meshes in the hologram plane. The optical field of each triangular mesh is usually calculated from its angular spectrum. According to the way the angular spectrum in the hologram plane is obtained, there are two approaches. In one approach, the angular spectrum of a triangle is first obtained in its local plane by taking two-dimensional (2D) discrete Fourier transform, and then the angular spectrum in the hologram plane is obtained by resampling the discrete local angular spectrum, considering the geometrical relationship between the hologram plane and the local plane [4–6]. In another approach, the angular spectrum in the hologram plane is directly obtained from an analytic formula of the angular spectrum of a reference triangle considering the geometrical relationship between the hologram and local planes and the spatial deformation of the triangle in the local plane [7–11].

In the first approach, the application of the texture mapping is straightforward. The only additional step for the texture mapping is to replace the uniform triangular mesh with the textured triangular mesh which is obtained from the texture image data as usual computer graphics. The textured triangular mesh is prepared in discrete form and its local angular spectrum is calculated by taking discrete Fourier transform in the local plane. Then the angular spectrum in the hologram plane is obtained by resampling the discrete local angular spectrum. In this method, however, the necessity of the angular spectrum resampling may cause artifacts and noise in the reconstruction.

In the second approach where the angular spectrum in the hologram plane is obtained directly from the analytic formula without resampling, the application of the texture mapping is not straightforward. The texture inside the triangular mesh may have arbitrary amplitude distribution and thus the analytic representation of its angular spectrum is not always possible. H.-G. Lim et al. has reported a method to apply the texture mapping to the fully-analytic method [12]. In this method, the texture image data inside the triangular mesh is expanded into a Fourier series. The angular spectrum of the uniform triangular mesh is shifted and accumulated in the hologram plane with weights of the Fourier series coefficients to obtain the angular spectrum of the textured triangular mesh, maintaining the fully analytic framework. Large number of the Fourier series coefficients, however, are required for high level of the details, which increases the computation time significantly. W. Lee et al. has reported improvement of the H.-G. Lim et al.’s method by approximating the mathematical formula to a shift-invariant form [13]. With this approximation, the weighted addition of the angular spectrums can be obtained from a single convolution operation, reducing the computation time. The error caused by the approximation, however, degrades the quality of the reconstruction especially when the triangular mesh is much slanted with respect to the hologram plane.

In this paper, we propose an efficient texture mapping method for fully-analytic triangular mesh based CGH. Instead of using Fourier series expansion like previous methods, the proposed method recursively divides the original triangular mesh into smaller ones and assigns uniform amplitudes to divided small triangular areas to maintain the fully-analytic framework without approximation. The recursive division is performed adaptively according to the complexity of the texture for each triangular mesh. With this adaptive division, the increase of the number of the triangular meshes can be minimized for given resolution of the texture. The calculation of the angular spectrum of divided small triangle is also performed effectively by using geometrical similarity between the divided triangle and the original triangle. From a few pre-calculated angular spectrum components of the original triangle, the angular spectrums of all divided sub-triangles are obtained by simple and fast manipulation without any interpolation or approximation. Therefore, the proposed method can generate the hologram of the textured object effectively without compromising the fully-analytic framework.

In the next section, we introduce the basic principles of texture mapping used in computer graphics and fully-analytic triangular mesh based CGH. Then, we explain the proposed algorithm including adaptive triangle division and efficient angular spectrum calculation of the divided sub-triangle. Finally, the simulation and optical experimental results are presented for the verification of the feasibility of the proposed method.

2. Review of texture mapping and fully-analytic triangle based CGH

2.1 Brief introduction to texture mapping in computer graphics

In computer graphics, the texture mapping is usually performed by preparing 2D texture image that contains all required textures of the 3D object and projecting individual triangular mesh of the 3D object onto the corresponding area in the texture pattern. According to the characteristics of the object such as shape and size, appropriate projection geometry is selected to minimize the artifact in the rendered result. Among commonly used projection geometry like planar, cylindrical, and spherical ones, the spherical projection is explained and used in this paper as an example. Using spherical coordinates system, a vertex at (r, θ, ϕ) of a triangular mesh is projected to a point in the texture pattern represented in Cartesian coordinates (u, v) by following relation.

u = 0 .5 + \frac{φ}{2 π}, v = 0 .5 + \frac{θ}{π},

where the elevation and azimuthal angles θ, and ϕ are in radian unit with a range [-π/2, + π/2] and [-π, + π], respectively. Three vertices of the triangle are projected by Eq. (1) and the area inside the projected vertices are used as the texture for the triangle as shown in Fig. 1.

Fig. 1 Concept of texture mapping (a) global coordinate of 3D object (b) UV coordinate of texture pattern

Download Full Size | PDF

2.2 Review of fully-analytic triangular mesh based CGH

In the fully-analytic triangular mesh based CGH, the hologram pattern U(x,y) of a single triangular mesh constituting the 3D object is obtained from its angular spectrum by

U (r_{x, y}) = \int \int_{- \infty}^{\infty} G (f_{x, y}^{}) \exp [j 2 π f_{x, y}^{T} r_{x, y}] d f_{x, y},

where r_x,y = [x, y]^T and f_x,y = [f_x, f_y]^T are the global position and spatial frequency vectors in the hologram plane, respectively. The angular spectrum G(f_x,y) in the hologram plane of a single triangular mesh is related to the angular spectrum G_l in the local plane containing the triangular mesh by

G (f_{x, y}) = G_{l} (f_{x l, y l}) exp [j 2 π f_{x l, y l, z l}^{T} c] \frac{f_{z l}}{f_{z}},

where f_xl,yl,zl = [f_xl, f_yl, f_zl]^T = Rf_x,y,z = R[f_x, f_y, f_z]^T, and f_xl,yl = [f_xl, f_yl]^T are spatial frequency vectors in the local plane [10,11,14]. The z components of the global and local spatial frequency vectors f_z and f_lz satisfy f_z = [(1/λ)²-f_x²-f_y²]^1/2, and f_lz = [(1/λ)²-f_lx²-f_ly²]^1/2, respectively. The 3 × 3 rotation matrix R is given by R = [u_xl^T u_yl^T u_zl^T]^T where u_xl, u_yl, and u_zl are 3 × 1 unit vectors of the local coordinate axes represented in the global coordinates. Together with the 3 × 1 shift vector c, the local position vector r_xl,yl,zl = [x_l, y_l, z_l]^T is related to the global position vector r_x,y,z = [x, y, z]^T by r_xl,yl,zl = Rr_x,y,z + c. Among possible choices of the shift vector c, c is chosen such that the local coordinate origin coincides with one of the three vertices of the triangle as shown in Fig. 2(a).

Fig. 2 Global and local coordinates of triangular mesh (a) transformation of the global coordinate to local coordinate (b) transformation of the local triangular mesh to the reference triangle

Download Full Size | PDF

Finally, the local angular spectrum is related to the angular spectrum of the reference triangle by

G_{l} (f_{x l, y l}) = \frac{G_{o} (A^{- T} {f_{x l, y l} - \frac{1}{λ} [\begin{matrix} u_{x l}^{T} \\ u_{y l}^{T} \end{matrix}] u_{c}})}{det (A)} = \frac{G_{o} (A^{- T} {f^{'}}_{x l, y l})}{det (A)} = \frac{G_{o} ({f^{'}}_{x r, y r})}{det (A)},

where G_o is the angular spectrum of the reference triangle whose analytic formula is known, and the 2 × 2 matrix A defines affine transform between the triangle in the local coordinates and the reference triangle as shown in Fig. 2(b) [10,11,14]. The direction of the carrier wave is denoted by the 3 × 1 unit vector u_c, and thus f′_xl,yl in Eq. (4) represents the local spatial frequency shifted by the carrier wave.

Therefore, from an analytic formula G_o of the angular spectrum of the reference triangle, the hologram pattern U(r_x,y) can be obtained by using Eqs. (2)-(4). Note that the global angular spectrum G over regular spatial frequency grid f_x,y in the global hologram plane is directly obtained from G_o by using Eqs. (2)-(4) without any resampling or interpolation. Also note that the angular spectrum of the reference triangle should be given by the analytic formula in order to fully exploit the benefit of this method. The analytic formula of the reference triangle is known only for the cases where the distribution inside the triangular area is uniform [7,8,10] or linear [11]. Therefore this basic method cannot be directly applied to the texture mapping application where the distribution inside the triangular area can be arbitrary.

3. Principle of the proposed method

In order to apply the texture mapping to the fully-analytic triangular mesh based CGH, we propose a method that uses adaptive mesh division and efficient angular spectrum calculation for divided meshes. The proposed method divides the original triangular mesh and assigns uniform amplitude to each of the divided ones, which enables the analytic representation of the reference angular spectrum. The mesh division is performed adaptively according to the amplitude variation inside the mesh area, minimizing the increase of the number of the divided meshes. The angular spectrums of the divided meshes are obtained efficiently from the angular spectrum components of the original mesh using the similarity relationship. In this section, the two main parts of the proposed method, i.e. adaptive mesh division and efficient angular spectrum calculation are explained.

3.1 Adaptive division of the triangular mesh

The proposed method divides the triangular mesh to represent higher details. The division process is adaptive in the sense that the division is performed only when the amplitude variation inside the current triangular mesh area is large. The division process is also recursive in the sense that the unit division process, i.e. bisecting all sides of the current triangle to obtain 4 similar triangles, is repeated until the maximum level of the division is reached or the amplitude variation inside the triangle is small. The concept of the proposed adaptive mesh division is illustrated in Fig. 3. In this way, the increase of the number of the divided triangular meshes can be minimized for the given texture and the similarity between the divided triangles and the original triangle can be maintained.

Fig. 3 Concept of adaptive and recursive mesh division

Download Full Size | PDF

Figure 4 shows the unit division process for a current triangle at a certain level of division. We obtain 4 triangles A, B, C, and D at the next level of division by bisecting the three sides of the current triangle. In Fig. 4, P₁′, P₂′, and P₃′ are the midpoints of the three sides.

Fig. 4 Unit division process (a) current triangle (b) divided triangles

Download Full Size | PDF

This unit division is performed only when the amplitude variation is large inside the current triangular area. Figure 5 shows two examples of the triangular meshes projected onto the UV coordinates of the texture image. When the projected triangular area has large amplitude variation as shown in Fig. 5(a), the triangle is divided into 4 triangles like right figure in Fig. 5(a). But when the current triangle has uniform value as shown in Fig. 5(b), the division process stops.

Fig. 5 Adaptive triangle division according to the texture data inside the triangle (a) An example of the case where the division is performed (b) an example of the case where the division is not performed

Download Full Size | PDF

In the proposed method, the determination of the division is performed by sampling a few points inside the current triangle. As shown in the middle figures in Figs. 5(a) and 5(b), 15 points including 3 vertices which uniformly sample the area of the triangular mesh are selected and the variation between the texture values on those points is measured. If the variation is higher than the threshold, then the triangle is divided into 4 ones as explained before. For each divided triangle, this 15 points sampling and the triangle division are performed again until the maximum recursive level of the division is reached.

Note that in our current implementation of the proposed method, we found that the division is not highly effective when the triangle mesh is much inclined such that the local plane containing the mesh is close to perpendicular to the hologram plane. In this case, the triangle projected onto the UV texture plane becomes severely obtuse triangle due to the spherical projection. Thus we excluded these triangles from further division in our implementation to maintain the effectiveness of the calculation.

3.2 Effective calculation of the angular spectrum of the divided triangular mesh

In this section, we explain a method to calculate the angular spectrum of the divided sub-triangle efficiently using similarity relationship between the divided sub-triangles and the original one. Due to the recursive side bisection used in the proposed method, all divided sub-triangles are similar to the original triangle regardless of the level of the recursive division. It is easy to show that the complex field of the divided triangle g_l⁽ⁿ⁾ in its local plane after n recursive divisions can be represented by

g_{l}^{(n)} (r_{x l, y l}) = g_{l, b a r e} ({(- 1)}^{k} 2^{n} (r_{x l, y l} - r_{g l}^{(n)}) + r_{g l}) \exp {j \frac{2 π}{λ} {([\begin{matrix} u_{x l}^{T} \\ u_{y l}^{T} \end{matrix}] u_{c})}^{T} r_{x l, y l}},

where n is the number of the recursive division, r_gl⁽ⁿ⁾ and r_gl are center of three vertices of the divided and the original triangles in local coordinates, respectively. The constant k is 0 when the divided triangle has the same orientation as the original one like triangles A, B, and C in Fig. 4(b) and it is 1 when the divided triangle has 180° rotated orientation like triangle D in Fig. 4(b). g_l,bare is the real-valued binary local function which is 1 inside the original triangle and 0 outside. The first term of Eq. (5) explains the appropriate shifting and scaling of g_l,bare to be the shape of the divided triangle |g_l⁽ⁿ⁾|. The phase distribution by the carrier wave on the divided triangle is shared with the original triangle because they are in the same local plane. The second term of Eq. (5) accounts for this phase distribution on the divided triangle. By taking Fourier transform to both sides of Eq. (5), we have

G_{l}^{(n)} (f_{x l, y l}) = \frac{1}{4^{n}} \exp [- j 2 π {f^{'}}_{x l, y l}^{T} (r_{g l}^{(n)} - \frac{r_{g l}}{{(- 1)}^{k} 2^{n}})] G_{l, b a r e} (\frac{{f^{'}}_{x l, y l}^{}}{{(- 1)}^{k} 2^{n}}),

where G_l,bare is the Fourier transform of g_l,bare with respect to r_xl,yl and f′_xl,yl is the local spatial frequency shifted by the carrier wave as defined in Eq. (4). Noting that G_l,bare(f′_xl,yl) = G_o(A^-T f′_xl,yl)/det(A) from Eq. (4), G_l⁽ⁿ⁾ of Eq. (6) can be written by

G_{l}^{(n)} (f_{x l, y l}) = \frac{1}{4^{n}} \exp [- j 2 π {f^{'}}_{x l, y l}^{T} (r_{g l}^{(n)} - \frac{r_{g l}}{{(- 1)}^{k} 2^{n}})] \frac{G_{o} (\frac{1}{{(- 1)}^{k} 2^{n}} {f^{'}}_{x r, y r}^{})}{\det (A)},

where f′_xr,yr = A^-Tf′_xl,yl as defined in the last equality of Eq. (4).

In Eq. (7), the angular spectrum of a reference triangle G_o may have different analytic form according to the shape of the reference triangle. In this paper, we choose a right triangle whose three vertices are located at (0,0), (1,0), and (1,1) and value is uniformly 1 inside the triangle and 0 outside. In this case, the angular spectrum can be obtained by [7,11]

\begin{array}{l} G_{o}^{} ({f^{'}}_{x r, y r}^{}) = \frac{\exp [- j 2 π {f^{'}}_{x r}] - 1}{{(2 π)}^{2} {f^{'}}_{x r} {f^{'}}_{y r}} + \frac{1 - \exp [- j 2 π ({f^{'}}_{x r} + {f^{'}}_{y r})]}{{(2 π)}^{2} {f^{'}}_{y r} ({f^{'}}_{x r} + {f^{'}}_{y r})} \\ = (T_{2} - T_{1}) + T_{1} \exp (- j T_{3}) - T_{2} \exp (- j T_{4}), \end{array}

where T₁, T₂, T₃, and T₄ are defined by

\begin{array}{l} T_{1} = \frac{1}{{(2 π)}^{2} {f^{'}}_{x r} {f^{'}}_{y r}}, T_{2} = \frac{1}{{(2 π)}^{2} {f^{'}}_{y r} ({f^{'}}_{x r} + {f^{'}}_{y r})}, \\ T_{3} = 2 π {f^{'}}_{x r}, T_{4} = 2 π ({f^{'}}_{x r} + {f^{'}}_{y r}) . \end{array}

Finally, from Eqs. (7)-(9), the local angular spectrum of the divided triangle can be given by

\begin{array}{l} G_{l}^{(n)} (f_{x l, y l}) = \frac{1}{\det (A)} \exp [- j 2 π {f^{'}}_{x l, y l}^{T} (r_{g l}^{(n)} - \frac{r_{g l}}{{(- 1)}^{k} 2^{n}})] \\ \times {(T_{2} - T_{1}) + T_{1} \exp [{(- 1)}^{k + 1} j \frac{T_{3}}{2^{n}}] - T_{2} \exp ({(- 1)}^{k + 1} j \frac{T_{4}}{2^{n}})} . \end{array}

In Eq. (10), it can be observed that most terms including T₁, T₂, T₃, and T₄, the shifted local spatial frequency grid f′_xl,yl, the affine transform matrix A, and the center of the original triangle r_gl are common for all sub-triangles divided from the original one, regardless of the recursive level n of the division and their locations due to similarity. Therefore, once we calculate those common terms for an original triangle, the local angular spectrums of all its sub-triangles can be calculated using Eq. (10) effectively. The flowchart of the proposed calculation is given in Fig. 6.

Fig. 6 Flow chart of the proposed method for each original triangular mesh

Download Full Size | PDF

Note that an alternative method one can consider is to divide the original triangle to M equal sub-triangles of the same size and shape without applying the adaptive division as presented in [8]. Then the angular spectrums of the sub-triangles are simply obtained from the discrete data of the angular spectrum of a single sub-triangle using shift relationship, i.e. by multiplying the angular spectrum of the sub-triangle with a linear phase distribution which accounts for the shift. In this case, the computation time of a single sub-triangle is small. However, the size of the sub-triangle is determined by the finest detail of the texture without considering the spatial distribution of the texture details over the original triangular area. As a result, in the area of coarse texture pattern, the sub-triangles are unnecessarily small, or the number of the sub-triangles are too large, causing waste of the total computation time.

In the proposed method using the adaptive division, the triangle is divided according to the spatial variation of the texture details, removing these waste caused by the unnecessarily large number of the sub-triangles. One possible drawback of the adaptive division is that the size of the final sub-triangles can be different, which might require the multiple addresses of the analytic formula of the reference angular spectrum. However, the proposed efficient calculation scheme summarized in Eq. (10) enables the acquisition of the angular spectrums of sub-triangles of different sizes from a few pre-calculated terms, without addressing the analytic formula additionally. Therefore the computation time of a single sub-triangle can be reduced comparable to the case of the M equal sub-division. Since the number of the total sub-triangles is much smaller in the proposed adaptive division than in the M equal sub-division, the total processing time is reduced in the proposed method. Another possible drawback of the proposed method is the computation time for decision of the division of the current triangle, i.e. measuring the amplitude variation inside the current triangle by probing a few points, which obviously increases the total computation time. However, our simulation and experimental results show that the effect of the reduced number of the final triangles by the adaptive division is more dominant and thus the total computation time remains much smaller than the conventional methods. We expect more efficient probing techniques would reduce the computation time further.

4. Simulation and experiment

We verified the proposed method by simulation and optical experiments. 3D objects used in the simulation and the experiments are a cube object and a teapot object shown in Fig. 7. The cube object consists of 24 triangular meshes and the teapot object has 756 meshes. In all optical experiments, a spatial light modulator of 8um pixel pitch and a green laser of 532nm wavelength are used. The generated hologram has 1920 × 1080 resolution and the displayed holographic images are located at 23cm from the spatial light modulator.

Fig. 7 3D objects used in the simulation and optical experiments. (a) cube object having 24 triangular meshes (b) teapot object having 756 triangular meshes

Download Full Size | PDF

First, the validity of Eq. (10) is verified. A test single triangle shown in Fig. 8(a) is divided into 16,384 sub-triangles by applying the unit division process explained in section 3.1 7 times. Then amplitudes are assigned to all sub-triangles and the angular spectrums are calculated by using two methods, i.e. conventional brute-force method which applies the method explained in section 2.2 to all sub-triangles and the proposed method which uses Eq. (10). Figure 8(b) and 8(c) show their reconstructions. It can be seen that they are exactly the same, proving that the angular spectrum calculated by Eq. (10) is precise.

Fig. 8 Comparison of reconstructed triangular mesh (a) original triangle (b) textured triangle in case of brute-force method (c) textured triangle in case of the proposed method

Download Full Size | PDF

Figure 9 shows the simulation and optical experiment results when the proposed method is applied to the entire 3D objects, cube and teapot. The maximum level of the recursive division was set to be n_max = 5. From Fig. 9, it can be seen that the texture pattern is well represented in the numerical and optical reconstructions of two objects as expected. Note that the deformation of the texture pattern on the surface of the reconstructed cube images does not come from the artifacts of the proposed method but from the characteristics of the spherical UV projection geometry explained in section 2.1. By using appropriate projection geometry, those deformations can be minimized.

Fig. 9 Simulation and optical experiment results of textured 3D object (a) texture pattern (b) simulation and optical reconstruction of the cube hologram generated by the proposed method (c) simulation and optical reconstruction of the teapot hologram generated by the proposed method

Download Full Size | PDF

Figure 10 and 11 are other examples of texture patterns and the optical reconstruction results. Two texture patterns shown in Fig. 10 represent different complexity. As can be seen in the results of Fig. 11, the proposed method is successful in synthesizing the hologram of 3D objects with texture patterns. It is also seen that the more the triangle is divided, the clearer the reconstruction represents the texture, which agrees with the expectation.

Fig. 10 Two texture patterns having different pattern density (a) low density texture pattern (b) high density texture pattern

Download Full Size | PDF

Fig. 11 Simulation and optical experiment results of textured 3D object for different maximum recursive division level n_max (a) cube object with low density texture pattern (b) cube object with high density texture pattern (c) teapot object with low density texture pattern (d) teapot object with high density texture pattern

Download Full Size | PDF

Next, the speed enhancement by the proposed method is verified. Holograms are calculated by three methods, i.e. the brute-force method, Kim et al.’s method reported in [8], and the proposed method. In the brute-force method, each original triangle is divided into small sub-triangles of the same size and the optical field corresponding to each sub-triangle is calculated using the method explained in section 2.2 without considering the similarity between the sub-triangles. In Kim et al.’s method of [8], each original triangle is divided in the same way, but the optical field calculation is performed using the similarity. In the proposed method, each original triangle is divided adaptively as explained in section 3.1 and the corresponding optical field is calculated using similarity by Eq. (10) in section 3.2. In all three methods, the size of the smallest sub-triangle is set to be the same for each original triangle, thus representing the same level of details of the texture pattern.

Figure 12 shows the total number of the sub-triangles. In case of the brute-force and Kim et al.’s [8] methods, the number of the sub-triangles is the same and it increases exponentially as the maximum level of the recursive division n_max increases. In case of the proposed method, owing to the adaptive division process, the number of the sub-triangle is much smaller. The effect is more obvious when the texture pattern has large uniform area as shown in Fig. 10(a). This reduced number of the sub-triangles contributes to the computation time reduction in combination with the efficient angular spectrum calculation using similarity.

Fig. 12 The number of the sub-triangles (a) cube object with low density texture pattern (b) cube object with high density texture pattern (c) teapot object with low density texture pattern (d) teapot object with high density texture pattern

Download Full Size | PDF

Note that it can be observed from comparison of cube and teapot results shown in Fig. 12 that the reduction of the number of the sub-triangles by the proposed method is more obvious in teapot object than the cube object. It is because the effectiveness of the proposed adaptive division is dependent on the relative texture complexity with respect to the size of the original triangle. The teapot object has many smaller triangles around the parts having complicated shape like handle, top hat, and the spout, while the cube object consists of larger triangles. For the given texture pattern, smaller triangles are projected on the smaller areas in the texture pattern, which results in higher chance of having large uniform area inside the triangle. Since the proposed adaptive division method is more efficient when the triangle has large uniform area, the proposed method shows better efficiency in the teapot object than in the cube object.

Also, note that the effectiveness of the proposed adaptive division method would decrease if the complexity of the texture pattern is extremely high overall without any uniform area. In this extreme case, the number of the divided sub-triangles of the proposed method will converge to that of the brute-force and Kim et al.’s [8] methods.

Figure 13 shows the total computation time comparison between three methods. From Fig. 13, it can be confirmed that the proposed method exhibits the speed enhancement over the brute-force method and Kim et al.’s method [8]. The speed enhancement of the proposed method over the brute-force method comes from two factors, i.e. the reduced number of the sub-triangles and the efficient angular spectrum calculation considering sub-triangle similarities. The speed enhancement over Kim et al.’s method [8] mainly comes from the reduced number of the sub-triangles. The speed enhancement of the proposed method over two conventional methods becomes larger when the level of the recursive division increases. Therefore the proposed method is especially efficient when the hologram needs to be generated with higher level of the texture details.

Fig. 13 Calculation time comparison (a) cube object with low density texture pattern (b) cube object with high density texture pattern (c) teapot object with low density texture pattern (d) teapot object with high density texture pattern

Download Full Size | PDF

Note that in all simulation and experimental results shown above, the phase distribution on each triangle is assumed to be linear which is given by a plane carrier wave. However, it is also possible to give different phase distribution so that the reconstructed image has wider viewing angle and desired angular brightness distribution, i.e. surface reflectance distribution for given illumination on the 3D object. It is accomplished by taking convolution of the angular spectrum obtained by Eq. (10) with a convolution kernel which contains the desired reflectance distribution, following the method presented in [14]. Figure 14 shows an experimental result with diffusive and specular reflection on the textured surface.

Fig. 14 Experimental result of textured teapot object with diffusive and specular reflection

Download Full Size | PDF

Note also that the continuous shading method [11] which allows each sub-triangle to have not only a uniform amplitude but also a linearly varying amplitude can be applied to the proposed method by simply replacing the angular spectrum of the uniform amplitude reference triangle given by Eq. (8) with that of the linearly varying amplitude reference triangle which is given in [11]. This continuous shading would benefit the adaptive mesh division of the proposed method especially when the texture pattern is not binary but has continuous tones. However, in this case, an efficient texture complexity probing technique should also be developed so that it can distinguish the uniform and linear variation of the texture pattern inside the current triangle from higher order variations to determine the necessity of further division of the current triangle. This is a topic of our further research.

5. Conclusion

We proposed a fully-analytic triangular mesh based method that synthesizes a hologram of 3D object with a texture pattern. In the proposed method, each triangular mesh is divided into sub-triangles and the amplitudes are assigned to the sub-triangles according to the texture pattern. The amplitude assigned to each sub-triangle is uniform, enabling the fully-analytic hologram generation without resampling or interpolation. The triangle division is performed by recursive application of unit division process, ensuring that all sub-triangles are similar to the original triangle. The level of the recursive division is determined adaptively according to the complexity of the texture pattern in order to minimize the increase of the number of the sub-triangles. The calculation of the angular spectrum of each sub-triangle is performed effectively by considering the similarity between the sub-triangle and the original triangle. As a result, the proposed method enables the efficient hologram generation of 3D objects having high texture details without sacrificing the precision. The feasibility of the proposed method was verified by simulations and optical experiments.

Funding

The Cross-Ministry Giga KOREA Project, Ministry of Science, ICT and Future Planning (MSIP), Korea [GK16D0100, Development of Telecommunications Terminal with Digital Holographic Table-top Display]; [GK16D0200, Development of Super Multi-View (SMV) Display Providing Real-Time Interaction].

References and Links

1. R. L. Cook, L. Carpenter, and E. Catmull, “The Reyes image rendering architecture,” Comput. Graph. 21(4), 95–102 (1987). [CrossRef]

2. S. Guha, Computer Graphics Through OpenGL: From Theory to Experiments (CRC, 2015).

3. F. M. Weinhaus and V. Devarajan, “Texture mapping 3D models of real-world scenes,” ACM Comput. Surv. 29(4), 325–365 (1997). [CrossRef]

4. K. Matsushima, “Computer-generated holograms for three-dimensional surface objects with shade and texture,” Appl. Opt. 44(22), 4607–4614 (2005). [CrossRef] [PubMed]

5. K. Matsushima, M. Nakamura, and S. Nakahara, “Novel techniques introduced into polygon-based high-definition CGHs,” in Biomedical Optics, OSA Technical Digest Series (Optical Society of America, 2010), paper JMA10.

6. K. Matsushima and S. Nakahara, “A high-definition full-parallax CGH created by the polygon-based method,” in Advances in Imaging, OSA Technical Digest Series (Optical Society of America, 2009), paper PDWB38.

7. L. Ahrenberg, P. Benzie, M. Magnor, and J. Watson, “Computer generated holograms from three dimensional meshes using an analytic light transport model,” Appl. Opt. 47(10), 1567–1574 (2008). [CrossRef] [PubMed]

8. H. Kim, J. Hahn, and B. Lee, “Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography,” Appl. Opt. 47(19), D117–D127 (2008). [CrossRef] [PubMed]

9. Y. Pan, Y. Wang, J. Liu, X. Li, and J. Jia, “Improved full analytical polygon-based method using Fourier analysis of the three-dimensional affine transformation,” Appl. Opt. 53(7), 1354–1362 (2014). [CrossRef] [PubMed]

10. J.-H. Park, H.-J. Yeom, H.-J. Kim, H. Zhang, B. Li, Y.-M. Ji, and S.-H. Kim, “Removal of line artifacts on mesh boundary in computer generated hologram by mesh phase matching,” Opt. Express 23(6), 8006–8013 (2015). [CrossRef] [PubMed]

11. J.-H. Park, S.-B. Kim, H.-J. Yeom, H.-J. Kim, H. Zhang, B. Li, Y.-M. Ji, S.-H. Kim, and S.-B. Ko, “Continuous shading and its fast update in fully analytic triangular-mesh-based computer generated hologram,” Opt. Express 23(26), 33893–33901 (2015). [CrossRef] [PubMed]

12. H.-G. Lim, N.-Y. Jo, and J.-H. Park, “Hologram synthesis with fast texture update of triangular meshes,” in Digital Holography and Three-Dimensional Imaging, OSA Technical Digest Series (Optical Society of America, 2013), paper DW2A.8.

13. W. Lee, D. Im, J. Paek, J. Hahn, and H. Kim, “Semi-analytic texturing algorithm for polygon computer-generated holograms,” Opt. Express 22(25), 31180–31191 (2014). [CrossRef] [PubMed]

14. H.-J. Yeom and J.-H. Park, “Calculation of reflectance distribution using angular spectrum convolution in mesh-based computer generated hologram,” Opt. Express 24(17), 19801–19813 (2016). [CrossRef] [PubMed]

Efficient texture mapping by adaptive mesh division in mesh-based computer generated hologram

Abstract

1. Introduction

2. Review of texture mapping and fully-analytic triangle based CGH

2.1 Brief introduction to texture mapping in computer graphics

2.2 Review of fully-analytic triangular mesh based CGH

3. Principle of the proposed method

3.1 Adaptive division of the triangular mesh

3.2 Effective calculation of the angular spectrum of the divided triangular mesh

4. Simulation and experiment

5. Conclusion

Funding

References and Links

Cited By

Figures (14)

Equations (10)

Optics Express