We propose a bar-type three-dimensional holographic head mounted display using two holographic optical elements. Conventional stereoscopic head mounted displays may suffer from eye fatigue because the images presented to each eye are two-dimensional ones, which causes mismatch between the accommodation and vergence responses of the eye. The proposed holographic head mounted display delivers three-dimensional holographic images to each eye, removing the eye fatigue problem. In this paper, we discuss the configuration of the bar-type waveguide head mounted displays and analyze the aberration caused by the non-symmetric diffraction angle of the holographic optical elements which are used as input and output couplers. Pre-distortion of the hologram is also proposed in the paper to compensate the aberration. The experimental results show that proposed head mounted display can present three-dimensional see-through holographic images to each eye with correct focus cues.
© 2015 Optical Society of America
Head mounted displays (HMDs) have been attracting growing attention recently. Two micro displays dedicated to the eyes of the user enable separated presentation of two parallax images to corresponding eyes, giving strong depth sensation by stereopsis. See-through optics enable the presentation of the stereoscopic three-dimensional (3D) images on top of the real environment, which is crucial in augmented reality applications.
In usual stereoscopic HMDs, however, the image presented to each eye is two-dimensional (2D) one formed at the conjugate plane of the micro display [1–9]. Thus each eye of the user focuses on the conjugate plane. On the contrary, the optical axes of two eyes converge at a distance which is given by the disparity between two parallax images presented to two eyes. Therefore, the focusing distance and the converging distance are different in the stereoscopic HMDs, which is called accommodation-vergence mismatch. The accommodation-vergence mismatch has been considered as a fundamental cause of the eye fatigue in 3D viewing [10, 11]. Besides the accommodation-vergence mismatch, the fixed distance of the conjugate plane also limits the presentation of the interactive objects in AR that has dynamic depth range. These drawbacks make users uncomfortable when using conventional stereoscopic HMDs.
In this paper, we propose a waveguide-type holographic HMD using two holographic optical elements (HOEs) and a spatial light modulator (SLM) for each eye. The proposed system delivers a holographic 3D image to each eye of the user instead of a 2D parallax image, removing the accommodation-vergence mismatch problem. The holographic 3D image is not confined in the conjugate plane of the SLM but it can be located at arbitrary distance from the eye, enabling realistic registration of the holographic 3D images to real scene in AR applications. The limitation of narrow viewing angle of usual holographic displays is relaxed in the proposed system since the relative position of the eye to the SLM is fixed in HMD configuration. The waveguide structure with two HOEs as input and output couplers enables the implementation of the proposed system in a thin and compact form factor and the presentation of the holographic 3D images in a see-through fashion. Note that the waveguide-type HMD configuration using two HOE couplers for each eye has already been reported [3–8]. What is unique in the system proposed in this paper is that the image delivered by the waveguide and presented to the eye is not a 2D image but a holographic 3D image, which removes the accommodation-vergence mismatch.
Main contribution of this paper is the analysis and the holographic compensation of the aberration in the waveguide-type holographic HMD system using HOE couplers. We find that the non-symmetric diffraction characteristics of the HOEs cause astigmatism of the holographic 3D images. This aberration is analyzed using ray tracing based on the coupled wave theory , and compensated by applying pre-distortion in the hologram generation for 3D images. The experimental results show that the proposed pre-distortion compensates the aberration successfully, displaying clear 3D holographic images in dynamic depth range. Although the holographic 3D HMD has been reported , to the authors’ best knowledge, the implementation with see-through waveguide-type configuration has not been proposed yet. The aberration analysis and the holographic compensation in the waveguide-type HMD with two HOE couplers are also original contribution of this paper.
In the following sections, we explain the system configuration, aberration analysis, holographic compensation, and the experimental verifications.
2. System configuration
Figure 1 shows the conceptual diagram of the proposed method. The system consists of a SLM, a bar-type waveguide, and two reflection-type HOEs. Linear grating is recorded in the HOEs so that they work as input and output couplers. After the laser light is diffracted by a SLM, it enters the waveguide and is incident on the in-coupling HOE in normal direction. The in-coupling HOE diffracts the incident normal light rays to an angle larger than the critical angle θc for the total internal reflection inside the waveguide. The light rays diffracted by the in-coupling HOE travel the waveguide through several internal reflections to reach the out-coupling HOE. At last, the light rays are diffracted by the out-coupling HOE to be directed to the normal direction and arrive at the user’s eye. A virtual image of the SLM is located at the distance of the optical path length that the light rays experience. The initial holographic 3D images formed by the SLM is also delivered to the space around the virtual image of the SLM. Therefore, one can see the holographic 3D images. Moreover, as the waveguide and the out-coupling HOE are transparent, the holographic 3D images are observed on top of the real environment, enabling see-through 3D presentation.
3. Aberration analysis and holographic compensation
In order to find the ray propagation characteristics in the waveguide, a ray tracing based on the H. Kogelnik’s coupled wave theory is performed. Figure 2 shows the vector diagrams of the in-coupling and out-coupling HOEs. With the coordinate system shown in Fig. 2, the grating vectors of two HOEs, i.e. Kin and Kout lie in the yz plane. The reference, signal and the grating vectors of the out-coupling HOE, i.e. kro,out, kso,out, and Kout are the mirror reflections of the signal, reference, and the grating vectors of the in-coupling HOE, i.e. kso,in, kro,in, and Kin with respect to the xy plane. In both HOEs, the reference and the signal beam vectors are related to the grating vector by
When a beam k1,air from the SLM in the air is incident on the in-coupling HOE through the waveguide, its wave vector k1,in in the in-coupling HOE region is represented byEq. (1) but given by Eq. (5) in order to meet the wave number condition, considering that the HOE is thin in z-direction so that the grating vector Kin is spread in the same direction as shown in Fig. 3.
The diffracted beam k2,in is refracted into the waveguide of the refractive index ng, guided inside the waveguide through total internal reflections, and refracted into the out-coupling HOE region. The wave vector k2,out of the incident beam on the out-coupling HOE is the mirror reflection of the k2,in with respect to the xy plane, i.e.
From Eqs. (2)-(9) and the fact that two grating vectors are in the mirror symmetry with respect to the xy plane, i.e. Kin,xy = Kout,xy, Kin,z = -Kout,z, it can be easily seen that the diffracted wave from the out-coupling HOE, k3,out is also the mirror reflection of the incident wave on the in-coupling HOE, k1,in. Therefore, the light ray from the SLM enters the user’s eye, maintaining its angle.
However, the optical path length that the light ray experiences inside the waveguide is dependent on the orientation of the plane of the incidence at the same angular deviation from the normal direction. Suppose that the light ray from the SLM is incident on the in-coupling HOE with the polar angle θin and the azimuthal angle φin in the in-coupling HOE region. As illustrated in Fig. 4, the diffracted wave by the in-coupling HOE, k2,in is given by Eqs. (3)-(5) with the transverse vector componentEquation (11) indicates that the angular deviation Δθ′of the diffracted light ray k2,in,θin = θin with respect to k2,in,θin = 0 = kso,in is dependent on the azimuthal angle φin of the incident ray k1,in. Therefore the incident beam cone around the normal direction (z-axis) is distorted around the k2,in,θin = 0 = kso,in after it is diffracted from the in-coupling HOE. This distortion makes the optical path length that the light rays experience in the waveguide be different for different φin, causing astigmatism in the final images.
More specifically, the ray path of a light ray emanating from a point (0, 0, -zo) in the SLM space with a polar angle θ and azimuthal angle φ can be calculated. For example, the transverse position of the ray on the bottom side of the waveguide when it exits the waveguide toward the user’s eye (xb,exit, yb,exit) is given by
Figures 5–7 show ray-tracing results for the verification of the analysis. In all simulations shown in Figs. 5–7, a glass waveguide of the thickness t = 3mm is assumed. The number of the internal reflection between the in-coupling and out-coupling HOEs is 5, i.e. m = 5, and the angle of the ray guided inside the waveguide is 70° with respect to the normal direction of the HOE. Figure 5 shows the footprint of the rays on the bottom side of the waveguide when they enter from the SLM and exit toward the eye. In Fig. 5, it can be found that the spread of the footprint of the exiting rays is larger in y-axis than in x-axis due to φ dependency of the angular deviation of Eqs. (11)-(14). Figures 6(a) and 6(b) show the rays coming to the eye. Note that the rays are extended to the virtual image space in Figs. 6(a) and 6(b) to identify the virtual image point clearly. From Fig. 6, it is obvious that the image point is focused at different distances in x and y directions, showing the astigmatism as expected.
The rays exiting from the waveguide toward the eye are mirror reflection of original rays from the point source as discussed before. Therefore the virtual image distances from the bottom side of the waveguide in x and y directions, i.e. zx, zy can be approximately estimated byFigure 7(a) shows the virtual image distances for different polar angles. As can be seen from Fig. 7(a), the astigmatism is dominant while spherical aberration is also identified. One notable thing is that the difference between directional image distances, i.e. Δz = zy- zx do not change with the point source distance zo as shown in Fig. 7(b). This property can also be verified from Eqs. (12)-(15).
In the proposed method, the constant difference Δz is simply compensated in the hologram generation process for the SLM by introducing the distance offset to y direction. For a 3D image which is composed of a set of 3D image points, the hologram is generated in the proposed method byEq. (16), compensating the astigmatism of the waveguide with two HOEs structure.
4. Experimental verification
Holographic aberration compensation for the waveguide type HMD with HOE input/output couplers and the presentation of the see-through holographic 3D images with correct focus cues were verified experimentally. In experiment, the HOEs were implemented by recording the linear gratings on the photopolymer sheet with λair = 532nm laser. The experimental setup to record the HOE is shown in Fig. 8. A polarizing beam splitter (PBS) divides a plane wave to the reference and signal beams which are incident on the photopolymer in normal direction and an angle larger than θc, respectively. Note that a right angle prism was attached on the photopolymer in the signal beam path in order to ensure that the signal beam angle inside the photopolymer is 70° which is larger than the critical angle θc of the waveguide glass – air boundary. In the configuration shown in Fig. 8, the half wave plate (HWP) 1 controls the power between reference beam and signal beam and the HWP 2 matches the polarization state of two beams to s-polarization. The power of the reference and the signal beams were adjusted to be the same as 0.5mW/cm2. The diffraction efficiency of the each of the recorded HOEs was measured to be around 34%.
Using the recorded HOEs, the experimental setup for the waveguide type holographic HMD was implemented. Figure 9 shows the implemented experimental setup. The waveguide is a glass slab with a thickness 3mm. The separation between two HOEs is about 49.5mm which corresponds to 5 times internal reflections of the guided light rays. The SLM used in the experiment is a reflection type with 8um pixel pitch. The experimental setup was configured such that the plane beam from the laser source passes through the in-coupling HOE and the waveguide as shown in Fig. 9 to be reflected by the SLM. The reflected and diffracted beams by the SLM are coupled into the waveguide by the in-coupling HOE. The coupled light is guided inside the waveguide and diffracted toward the camera or the user’s eye by the out-coupling HOE.
The astigmatism aberration and its holographic compensation were first verified experimentally. A hologram of a mesh grid pattern at 60cm distance behind the SLM was generated without aberration compensation and loaded in the SLM. Figure 10(a) and 10(b) show the images captured at different focal distances of the camera. As can be observed in Figs. 10(a) and 10(b), the horizontal and vertical lines of the reconstructed mesh grid pattern are focused at different focal distances of the camera, revealing the astigmatism caused by the HOEs and the waveguide structure clearly.
Figure 10(c) shows the reconstructed image when the holographic compensation is applied. In Fig. 10(c), the camera was focused at 65cm distance. In the holographic compensation, the distance difference Δz was set to 17.7cm. It is obvious from Fig. 10(c) that the reconstructed image becomes much clearer by the proposed holographic compensation. Figure 11 shows another example of the effect of the aberration and its compensation by the proposed method, proving its feasibility.
Figure 12 shows the spreads of a single point reconstructed with and without proposed compensation. The intensity profiles in Fig. 12 were obtained by summing columns or rows of the captured images and normalizing them. The point reconstructed with the proposed compensation shows narrow spread both in horizontal and vertical directions as shown in Fig. 12(a). When the proposed compensation is not applied, the reconstructed point shows asymmetric spread according to the focusing distance of the camera as shown in Figs. 12(b)-12(d), failing to be focused sharply both in the horizontal and vertical directions simultaneously. Comparing Figs. 12(a) and 12(b), it can be observed that the full-width-at-half-maximum (FWHM) of the reconstructed point is reduced from 19(x) × 26(y) pixels to 6(x) × 6(y) pixels by the proposed compensation, showing enhancement over 3 times. In our experimental setup, the 6 pixel FWHM of Fig. 12(a) corresponds approximately to 522um width at 56.5cm distance from the camera, which indicates the spatial resolution of the proposed system roughly.
Figure 13 shows the reconstructed images at different distances from the camera. In all images, the proposed holographic compensation was applied with constant Δz = 17.7cm. The red object in Fig. 13 is a real object at 32cm distance which is seen through the waveguide and the out-coupling HOE. From Fig. 13, it can be seen that the proposed waveguide-type holographic HMD can present a see-through holographic 3D images in a wide depth range.
Figure 14 shows reconstructed images at different distances at the same time. The holographic cube and the teapot images are displayed at the same distances as the real red and black objects, respectively. The depth difference between the teapot and the cube images is 25cm. From Fig. 14, it can be seen that the reconstructed holographic images are focused and blurred in the same way as the real objects at the same distances, showing the 3D nature of the reconstructed holographic images successfully.
In this paper, we proposed a waveguide type 3D holographic HMD using HOE input-output couplers with aberration compensation. The astigmatism aberration caused by the asymmetric diffraction of the HOEs for a given angular deviation of the light from the SLM is analyzed and its simple compensation by introducing distance offset for x and y directions is proposed. The experimental results show that the proposed system can present holographic 3D images in a see-through fashion clearly in a wide depth range.
This research was partly supported by 'The Cross-Ministry Giga KOREA Project' of The Ministry of Science, ICT and Future Planning, Korea. [GK15D0100, Development of Telecommunications Terminal with Digital Holographic Table-top Display]. This research was also partly supported by 'The Cross-Ministry Giga KOREA Project' of The Ministry of Science, ICT and Future Planning, Korea. [GK15D0200, Development of Super Multi-View (SMV) Display Providing Real-Time Interaction]. This research was also partly supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2015-R0992-15-1008) supervised by the IITP (Institute for Information & communications Technology Promotion).
References and links
1. Y. Amitai, “A two-dimensional aperture expander for ultra-compact, high-performance head-worn displays,” SID Symposium Digest Of Technical Papers. 36(1), 360–363 (2005). [CrossRef]
2. T. Levola, “Diffractive optics for virtual reality displays,” J. Soc. Inf. Disp. 14(5), 467–475 (2006). [CrossRef]
3. H. Mukawa, K. Akutsu, I. Matsumura, S. Nakano, T. Yoshida, M. Kuwahara, K. Aiki, and M. Ogawa, “A full color eyewear display using holographic planar waveguides,” SID Symposium Digest of Technical Papers. 39(1), 89–92 (2008).
4. H. Mukawa, K. Akutsu, I. Matsumura, S. Nakano, T. Yoshida, M. Kuwahara, and K. Aiki, “A full-color eyewear display using planar waveguides with reflection volume holograms,” J. Soc. Inf. Disp. 17(3), 185–193 (2009). [CrossRef]
5. R. Shi, J. Liu, H. Zhao, Z. Wu, Y. Liu, Y. Hu, Y. Chen, J. Xie, and Y. Wang, “Chromatic dispersion correction in planar waveguide using one-layer volume holograms based on three-step exposure,” Appl. Opt. 51(20), 4703–4708 (2012). [CrossRef] [PubMed]
6. J. Piao, G. Li, M. Piao, and N. Kim, “Full Color Holographic Optical Element Fabrication for Waveguide-type Head Mounted Display Using Photopolymer,” J. Opt. Soc. Korea 17(3), 242–248 (2013). [CrossRef]
7. M. L. Piao and N. Kim, “Achieving high levels of color uniformity and optical efficiency for a wedge-shaped waveguide head-mounted display using a photopolymer,” Appl. Opt. 53(10), 2180–2186 (2014). [CrossRef] [PubMed]
8. L. Xia, K. Xu, Z. Wu, Y. Hu, Z. Li, Y. Wang, and J. Liu, “A green-color portable waveguide eyewear display system,” Proc. SPIE 8913, 89130W (2013). [CrossRef]
9. J. Han, J. Liu, X. Yao, and Y. Wang, “Portable waveguide display system with a large field of view by integrating freeform elements and volume holograms,” Opt. Express 23(3), 3534–3549 (2015). [CrossRef] [PubMed]
10. J. Hong, Y. Kim, H.-J. Choi, J. Hahn, J.-H. Park, H. Kim, S.-W. Min, N. Chen, and B. Lee, “Three-dimensional display technologies of recent interest: principles, status, and issues [Invited],” Appl. Opt. 50(34), H87–H115 (2011). [CrossRef] [PubMed]
11. S. Park, J. Yeom, Y. Jeong, N. Chen, J.-Y. Hong, and B. Lee, “Recent issues on integral imaging and its applications,” J. Inform. Display 15(1), 37–46 (2014). [CrossRef]
12. H. Kogelnik, “Coupled wave theory for thick hologram gratings,” Bell Labs Tech. J. 48(9), 2909–2947 (1969). [CrossRef]