Open Access
Add to BibSonomy
Abstract
We demonstrate a multi-focal-plane see-through near-eye display with effective focus cues enabled by wavelength multiplexing. A spectral notch filter is implemented as the wavelength-sensitive depth separation element. The vergence-accommodation conflict can be mitigated with the proposed design without space- or time-multiplexing. Another design of a dual-focus projection module for the waveguide-type augmented reality devices using wavelength-multiplexing is also presented.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
With rapid progress in electronics and optics, virtual reality (VR) and augmented reality (AR) devices have started a new round of innovation toward next-generation mobile platforms. To deliver immersive and realistic user experience, such a head-mounted display (HMD) system needs to provide both high-quality images and real three-dimensional (3D) sensations [1,2]. The human visual system perceives the depth information through both psychological and physiological cues. Proper psychological cues [3] can be achieved by advanced flat-panel displays with high resolution, high contrast ratio, fast frame rate, and good color accuracy. However, regarding the physiological cues, the state-of-art devices can only provide proper convergence cue but not the according accommodation cue, causing headaches and eye strain due to the vergence-accommodation conflict (VAC) [4,5]. VAC exists because the two eyes focus (accommodate) to a physical display depth while they rotate and align (converge) to another simulated depth generated with visual disparity, which is behind or before the physical display. Unlike in the real world, the correlation of these two cues are usually decoupled in 3D HMDs with a fixed physical depth, resulting in the VAC issue.
Several methods and designs have been developed to resolve this conflict [6], including but not limited to light field displays (also referred to as multiscopic displays, integral imaging) [7–12], multifocal or volumetric displays [13–20], and holographic displays [21–23]. Correct or pseudo-correct accommodation cues can be provided with a multifocal configuration such that the difference between the vergence and accommodation cues is significantly reduced [24,25]. These previous works generally function in a time- or space-multiplexing manner, where the frame rate or resolution is compromised for a proper accommodation cue. If the screen-door effect is considered, then fewer existing AR/VR devices can afford a decreased resolution, especially those demanding a large field-of-view (FOV). Similarly, the motion picture response time [26] will increase significantly if the frame rate is reduced in half, which would cause noticeable image blur when displaying fast moving objects. To avoid image quality degradation, a simple method is to implement a compact dual-focus HMD [27] based on polarization multiplexing using Pancharatnam-Berry phase lens [28]. However, because there are only two orthogonal polarization states, only two focal planes can be generated.
In this paper, we propose to generate multiple focal planes in HMDs through wavelength multiplexing, using another dimension of light as the information channel to expand the 2D images into 3D. Due to the degenerate spectral response of human vision system, different sets of primary colors can cover the same color space. Compared to prior methods, wavelength multiplexing intrinsically allows more than two focal planes without losing resolution or frame rate. To prove concept, we build a benchtop device based on the birdbath architecture but with an extra green spectral notch filter. We also analyze the implementation of full-color operation with a multi-notch filter based on commercially available optical multi-layer coatings. Finally, we discuss practical limitations and potential improvement of the proposed wavelength multiplexing method for overcoming the VAC issue.
2. System design
Figure 1 depicts the system configuration of our proposed see-through multi-focal-plane display. This design has a modified birdbath optical structure, which has been widely used as optical combiner and projection units in AR devices. Here two laser projectors and a projection screen serve as the graphic generation unit, which can be replaced by a liquid-crystal-on-silicon (LCoS) display as well. The primary colors in the two laser projectors are slightly different (λA and λB) such that λA lies in the reflection band of the notch filter, while λB is transmitted. The beam splitter functions not only as the see-through optical combiner for the background light but also as part of the folding optics generating the virtual images at different depths.
Figures 2(a) and 2(b) depict the unfolded optical layout of the proposed system for λA and λB, respectively. By changing the distance between the notch filter and mirror as Fig. 1 depicts, the optical power of the effective eyepiece for λB can be tuned. In this manner, the image contents with different wavelengths are projected to different depths. By adding more wavelengths and notch filters, more than two focal planes can be generated.
3. Experiment
To demonstrate the feasibility of multi-focal-plane near-eye displays based on wavelength multiplexing, we constructed the proposed design (Fig. 1) using off-the-shelf optics and laser projectors. In our experiments, two laser projectors with different green primary wavelengths are employed as part of the image generation unit that also includes a scattering projection screen. The laser projector A has a green laser diode with wavelength λA=532 nm, while the laser diode in projector B emits light at λB=517 nm. A notch filter with high reflectance at 532 nm but high transmittance at 517 nm was utilized to distinguish the wavelength information in the image content. The measured spectrum of laser projectors and transmittance of the notch filter are plotted in Fig. 3. The mirror was placed 2 cm behind the notch filter. The focal length of the refractive lens in our setup is 20 cm, and the distance between the projection screen and the refractive lens is 8 cm. Figures 4(a) and 4(b) shows two photos captured through the experimental setup with different focal planes, one at ∼2 m and the other at ∼1 m away from the beam splitter. We placed two real objects at these two distances, a flower and a monkey, to construct the exterior environment for a clear depth demonstration. The photographs were taken with Sigma 28–300 mm F3.5–6.3 lens on a Canon 6D DSLR camera for a narrow depth of field. Although there are apparent laser speckles due to the coherent light source, the multi-focal functionality is clearly demonstrated within a 10° FOV.
Fig. 4. Photos captured through the near-eye display prototype, where the camera focuses at around (a) 2 m and (b) 1 m in front of the beam splitter.
4. Discussion
An apparent concern of wavelength multiplexing is the system color performance. Because the primary wavelengths are different for each depth, the system color gamut is defined by the intersection of all color gamuts at every depth. Figure 5 shows the color gamut from two exemplary laser projectors with commercially available primary wavelengths (442 nm, 520 nm, and 635 nm) and (465 nm, 532 nm, and 660 nm). Since laser projectors have intrinsic better color purity than other light sources, the overlapping color area of the two laser projectors can still cover most of the DCI-P3 color gamut. The emitting wavelength of laser diodes can be tuned by changing gain medium, intracavity filter, and working temperature. As a promising light source for HMD applications, various laser diodes dedicated to displays are also under development [29].
Another critical issue is the angular dependent stopband location of the spectral notch filter, which is caused by the deviation of optical path length in dielectric optical layers at oblique incidence. Although optical coating technologies [30] are already well developed for various spectral filters, the design and fabrication of angle-insensitive coating for a multi-notch filter are still challenging and expensive. To address this issue, we propose to utilize the etendue conservation in optical systems. The angle of incidence on a relatively large notch filter can be restricted to a small range in an AR system while a large FOV is maintained at a relatively small exit pupil.
For example, Figs. 6(a)–6(d) present an applicable dual-focus optical design of a projection module in the AR displays. The FOV at exit pupil is from − 20° to 20°, while the largest angle of incidence on the notch filter is around 9.5°. Apparently, the dual-focus projection module can function well under the illumination from two sets of RGB laser diodes in a time sequential manner, where the frame rate requirement can be demanding. Instead, we propose anther illumination method using a digital micromirror (DMD) device conjugated to the LCoS panel, as shown in Fig. 7(a). The bistable DMD can tune the spectrum ratio between two sets of wavelengths in each pixel of the graphics, while the LCoS is responsible for amplitude modulation, as illustrated in Fig. 7(b). In this manner, the two images with independent contents can be generated simultaneously.
Fig. 6. Visualization of (a) a non-sequential (TracePro) and (b) the according unfolded sequential (CodeV) ray-tracing design of the proposed wavelength-multiplexed dual-focus projection unit for AR display. Virtual image distance: (upper) close to infinity; (lower) 1.5 m.
5. Conclusion
We have designed and demonstrated an AR display system with two focal planes based on wavelength multiplexing using a spectral notch filter. The prototype clearly verifies the feasibility of resolving the VAC issue in HMDs through wavelength multiplexing. Although there are several practical limitations in the proposed system, the wavelength multiplexing method is proven to be applicable for multi-focal-plane displays without sacrificing resolution or frame rate.
Funding
Intel Corporation.
References
1. J. Geng, “Three-dimensional display technologies,” Adv. Opt. Photonics 5(4), 456–535 (2013). [CrossRef]
2. B. Lee, “Three-dimensional displays, past and present,” Phys. Today 66(4), 36–41 (2013). [CrossRef]
3. S. J. Watt, K. Akeley, M. O. Ernst, and M. S. Banks, “Focus cues affect perceived depth,” J. Vis. 5(10), 7 (2005). [CrossRef]
4. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008). [CrossRef]
5. M. Mon-Williams, J. P. Warm, and S. Rushton, “Binocular vision in a virtual world: visual deficits following the wearing of a head-mounted display,” Ophthalmic Physiol. Opt. 13(4), 387–391 (1993). [CrossRef]
6. G. Kramida, “Resolving the vergence-accommodation conflict in head-mounted displays,” IEEE Trans. Vis. Comput. Graph. 22(7), 1912–1931 (2016). [CrossRef]
7. H. Arimoto and B. Javidi, “Integral 3D imaging with digital reconstruction,” Opt. Lett. 26(3), 157–159 (2001). [CrossRef]
8. S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express 12(3), 483–491 (2004). [CrossRef]
9. T. Zhan, Y. H. Lee, and S. T. Wu, “High-resolution additive light field near-eye display by switchable Pancharatnam–Berry phase lenses,” Opt. Express 26(4), 4863–4872 (2018). [CrossRef]
10. F. C. Huang, K. Chen, and G. Wetzstein, “The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues,” ACM Trans. Graph. 34(4), 60 (2015). [CrossRef]
11. J. Y. Wu, P. Y. Chou, K. E. Peng, Y. P. Huang, H. H. Lo, C. C. Chang, and F. M. Chuang, “Resolution enhanced light field near eye display using e-shifting method with birefringent plate,” J. Soc. Inf. Disp. 26(5), 269–279 (2018). [CrossRef]
12. Z. Qin, P.-Y. Chou, J.-Y. Wu, C.-T. Huang, and Y.-P. Huang, “Resolution-enhanced light field displays by recombining subpixels across elemental images,” Opt. Lett. 44(10), 2438–2441 (2019). [CrossRef]
13. E. Downing, L. Hesselink, J. Ralston, and R. Macfarlane, “A three-color, solid-state, three-dimensional display,” Science 273(5279), 1185–1189 (1996). [CrossRef]
14. G. D. Love, D. M. Hoffman, P. J. W. Hands, J. Gao, A. K. Kirby, and M. S. Banks, “High-speed switchable lens enables the development of a volumetric stereoscopic display,” Opt. Express 17(18), 15716–15725 (2009). [CrossRef]
15. Q. Chen, Z. Peng, Y. Li, S. Liu, P. Zhou, J. Gu, J. Lu, L. Yao, M. Wang, and Y. Su, “Multi-plane augmented reality display based on cholesteric liquid crystal reflective films,” Opt. Express 27(9), 12039–12047 (2019). [CrossRef]
16. Y. H. Lee, F. Peng, and S. T. Wu, “Fast-response switchable lens for 3D and wearable displays,” Opt. Express 24(2), 1668–1675 (2016). [CrossRef]
17. S. Liu and H. Hua, “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010). [CrossRef]
18. Y. H. Lee, G. Tan, T. Zhan, Y. Weng, G. Liu, F. Gou, F. Peng, N. V. Tabiryan, S. Gauza, and S. T. Wu, “Recent progress in Pancharatnam-Berry phase optical elements and the applications for virtual/augmented realities,” Opt. Data Process. Storage 3(1), 79–88 (2017). [CrossRef]
19. C.-K. Lee, S. Moon, S. Lee, D. Yoo, J.-Y. Hong, and B. Lee, “Compact three-dimensional head-mounted display system with Savart plate,” Opt. Express 24(17), 19531–19544 (2016). [CrossRef]
20. S. Lee, C. Jang, S. Moon, J. Cho, and B. Lee, “Additive light field displays: realization of augmented reality with holographic optical elements,” ACM Trans. Graph. 35(4), 1–13 (2016). [CrossRef]
21. K. Wakunami, P.-Y. Hsieh, R. Oi, T. Senoh, H. Sasaki, Y. Ichihashi, M. Okui, Y.-P. Huang, and K. Yamamoto, “Projection-type see-through holographic three-dimensional display,” Nat. Commun. 7(1), 12954 (2016). [CrossRef]
22. G. Li, D. Lee, Y. Jeong, J. Cho, and B. Lee, “Holographic display for see-through augmented reality using mirror-lens holographic optical element,” Opt. Lett. 41(11), 2486–2489 (2016). [CrossRef]
23. E. Moon, M. Kim, J. Roh, H. Kim, and J. Hahn, “Holographic head-mounted display with RGB light emitting diode light source,” Opt. Express 22(6), 6526–6534 (2014). [CrossRef]
24. S. Liu, Y. Li, P. Zhou, Q. Chen, S. Li, Y. Liu, Y. Wang, and Y. Su, “Full-color multi-plane optical see-through head-mounted display for augmented reality applications,” J. Soc. Inf. Disp. 26(12), 687–693 (2018). [CrossRef]
25. Y. H. Lee, G. Tan, K. Yin, T. Zhan, and S. T. Wu, “Compact see-through near-eye display with depth adaption,” J. Soc. Inf. Disp. 26(2), 64–70 (2018). [CrossRef]
26. F. Peng, H. Chen, F. Gou, Y. H. Lee, M. Wand, M. C. Li, S. L. Lee, and S. T. Wu, “Analytical equation for the motion picture response time of display devices,” J. Appl. Phys. 121(2), 023108 (2017). [CrossRef]
27. G. Tan, T. Zhan, Y. H. Lee, J. Xiong, and S. T. Wu, “Polarization-multiplexed multi-plane display,” Opt. Lett. 43(22), 5651–5654 (2018). [CrossRef]
28. T. Zhan, Y. H. Lee, G. Tan, J. Xiong, K. Yin, F. Gou, J. Zou, N. Zhang, D. Zhao, J. Yang, S. Liu, and S. T. Wu, “Pancharatnam–Berry optical elements for head-up and near-eye displays [Invited],” J. Opt. Soc. Am. B 36(5), D52–D65 (2019). [CrossRef]
29. M. Murayama, Y. Nakayama, K. Yamazaki, Y. Hoshina, H. Watanabe, N. Fuutagawa, H. Kawanishi, T. Uemura, and H. Narui, “Watt-Class Green (530 nm) and Blue (465 nm) Laser Diodes,” Phys. Status Solidi A 215(10), 1700513 (2018). [CrossRef]
30. M. Scherer, U. Schallenberg, H. Hagedorn, W. Lehnert, B. Romanov, and A. Zöller, “High performance notch filter coatings produced with PIAD and magnetron sputtering,” Proc. SPIE 7101, 71010I (2008). [CrossRef]
