In this paper we propose an optical see-through multi-plane display with reverse-mode polymer-stabilized liquid crystal (PSLC). Our design solves the problem of accommodation-vergence conflict with correct focus cues. In the reverse mode PSLC system, power consumption could be reduced to ~1/(N-1) of that in a normal mode system if N planes are displayed. The PSLC films fabricated in our experiment exhibit a low saturation voltage ~20 Vrms, a high transparent-state transmittance (92%), and a fast switching time within 2 ms and polarization insensitivity. A proof-of-concept two-plane color display prototype and a four-plane monocolor display prototype were implemented.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Augmented reality (AR) displays have been developed rapidly in recent years with widespread applications [1, 2]. However conventional head mounted display (HMD) products mainly employ stereoscopic 3D technique, which results in an accommodation-vergence conflict problem [3, 4]. To solve this problem several methods have been proposed, such as holographic display [5, 6], volumetric display [7–11], integral imaging display [4, 12] and super-multi-view display [13, 14]. Among those technologies, the multi-plane volumetric 3D display is one candidate to display a 3D image with acceptable computation load by taking the advantage of the mature 2D display technologies. The multi-plane volumetric 3D display has been considered as a true 3D display, since it is capable of constructing a 3D scene with correct depth information.
To realize a multi-plane HMD, many approaches have been proposed [7–11, 15–22]. For example, Hua et al. designed a multi-focal-plane HMD using a liquid lens [11,15]. The same research group later realized a six-plane HMD operated at 60 Hz using a fast DMMD . Lin et al. proposed a vary-focal-plane HMD by using a liquid crystal lens . Lee et al. designed a compact two-plane HMD using a birefringent savart plate . Wu et al. proposed multi-plane displays based on a fast switching polarization converter [18, 19] and based on polymer-stabilized cholesteric scattering films . In our previous work we proposed a multi-plane optical see-through HMD using a stack of normal mode polymer-stabilized liquid crystal (PSLC) scattering shutters . However, by employing the normal mode PSLC films, at one time only one PSLC film was in the voltage-off scattering state, while all the other films were in the voltage-on transparent state. Thus the power consumption was relatively high.
In this paper we propose a multi-plane optical see-through HMD with a stack of reverse mode PSLC scattering shutters. The reversed mode PSLC films exhibit an optically transparent state when no voltage is applied, and shows a scattering state with a sufficiently high voltage. Thus only one of the N PSLC films requires applied voltage at one time, and the power consumption is ~1/(N-1) of that in the normal mode PSLC system. The reverse mode PSLC shutters have a relatively low driving voltage with a saturation voltage of about 20 Vrms, and offer fast switching time (< 2 ms). The transmittance of the clear state is about 92%, which is higher than that of the normal mode PSLC shutter (86%) in our previous work. The reverse mode PSLC films show a polarization insensitive optical property, therefore, our system does not require any polarization optical components, and thus saves more optical power. In this paper we implement a proof-of-concept two-plane color prototype and a four-plane monocolor prototype.
2. System design
As illustrated in Fig. 1(a), there are four major components in our system: a high speed projector, a stack of reverse mode PSLC films, an ocular lens and an optical combiner such as a beam splitter (BS) or a half mirror. The reverse mode PSLC scattering shutters are electrically-controllable films which can be fast switched from an optical clear state to a scattering state with applied voltage, as shown in Fig. 1(b) 1(c). The high speed projector projects 2D slice images of the 3D object onto the PSLC films in time sequence. By controlling the on/off states of the PSLC films and the 2D image frames sequentially, a miniature 3D image of the object can reconstructed on these PSLC scattering shutters.
Each reverse mode PSLC film is located at a proper distance from the ocular lens. So in order to form a clear and sharp image on every PSLC scattering shutter, it is required that the light from the projector is collimated. And then the viewer would see both the images magnified by the ocular lens and the real world scene through the combiner. Because of the collimated image source, all the images projected on the PSLC films are of the same size; moreover, as the light is scattered by the PSLC films, the chief rays are always kept parallel to the optical axis of the ocular lens, forming telecentricity in the object space. As a result, the images at different planes have the same field of view (FOV) when observed at the back focal plane position . Also the angular resolution of different planes remains constant and the corresponding pixels on different depth planes overlap with each other in our system. To realize the multi-plane function in our system, it is essential to have a high speed collimated image source and a stack of reverse mode PSLC films with fast switching time, high-clear state transmittance and low scattering-state transmittance.
3. Reverse mode PSLC scattering shutters
Polymer stabilized liquid crystal films have been developed for display and light shutter applications [23–25]. In the normal mode operation a PSLC shutter is in a scattering state without applied voltage; and turns into a transparent state with applied voltage, as the positive LC directors are switched almost parallel to the electric field. On the contrary, the reverse mode PSLC film is in a transparent state without applied voltage and in an opaque state with applied voltage, as shown in Fig. 2. In the voltage-off state, the negative LC directors are uniformly aligned perpendicular to the substrate due to the vertical alignment. For normal incident light, since the ordinary refractive index of the LC no is close to that of the polymer np, the light passes through the PSLC film without scattering. When an external vertical electric field is applied, the negative LC directors got reorientated and become perpendicular to the electric field, with random azimuth angles in the x-y plane as shown in Fig. 2. Because of the refractive index mismatch between the different LC domains, the incident light is strongly scattered.
3.2 Monomer type effect
The electro-optic properties of the reverse mode PSLC scattering shutters are governed by many factors, such as monomers, LC host and curing conditions [26, 27]. Especially monomers play an important roles in the LC/polymer composite. In our experiment, we prepared two samples to investigate the effect of monomer types. We chose a host liquid crystal (LC) HNG715611-000 (Jiangsu Hecheng Advanced Materials Co. Ltd, = −12.2, no = 1.493, ne = 1.646) with negative dielectric anisotropy. Two different UV curable monomer TMPTA (Aldrich) and RM257 (Jiangsu Hecheng Advanced Materials Co. Ltd) were added to two samples, respectively. Each sample was composed of 6 wt% monomer (RM257 or TMPTA), 94% wt% LC and a small amount of UV photoinitiator. The two samples were filled into two identical 9 μm indium-tin-oxide (ITO) coated cells with vertical alignment, respectively, via capillary action in the dark environment. Then the cells were polymerized by UV light with irradiation of 15 mW/cm2 for about 20 minutes at room temperature.
The transmittance of the samples was measured at 532 nm with a green laser. In Fig. 3 the transmittance is defined as the ratio of transmitted light intensity to incident light intensity. The TMPTA PSLC film exhibits about 80% transmittance in the optical clear state and about 10% transmittance in the scattering state, while the RM257 PSLC scattering shutter exhibits about 92% transmittance in the optical clear state and about 5% transmittance in the scattering state, as illustrated in Fig. 3(a). Because of better refractive index match, the RM257 PSLC scattering shutter shows a higher transmittance in the clear state, which indicates higher optical efficiency and less cross talk between different image planes.
The saturation voltage of TMPTA PSLC film is about 60 Vrms, while the RM257 PSLC film is only about 30 Vrms. The TMPTA is a tri-functional monomer, and the RM257 is a di-functional monomer. After polymerization a stronger network is formed in a TMPTA PSLC film [25, 26]. And this greatly increases the interaction between the polymer network and LC molecules. So the driving voltage of the TMPTA PSLC film is much higher than that of the RM257 PSLC film.
We also measured the transmittance for the RM257 and TMPTA PSLC films in the clear and scattering states as the polarization direction of the incident light was varied. As shown in Fig. 3(b), the transmittance of the clear and scattering states for both samples does not change noticeably as the polarization angle of incident light changes, indicating our reverse mode PSLC films are polarization insensitive. In the voltage-off state, the LC molecules are well-aligned homeotropically, so the normal incident light propagates along the optic axes of the LC directors and encounters the same refractive index no regardless of polarization. When an external voltage is applied, the LC directors are reorientated parallel to the substrate. However, in one domain the LC molecules tilt toward in x direction while in another domain they may tilt toward y direction. So this explains why the transmittance of the scattering state is insensitive to polarization direction variation. By using the polarization insensitive PSLC scattering shutters, our system does not require any polarization optical component, and thus is more energy efficient.
The response times of PSLC films directly determines the maximum number of planes that could be displayed in the system. Figure 4 shows the measured response time of the two PSLC scattering shutters using a 30 Hz alternative-current square-wave voltage. The rise time of the TMPTA PSLC shutter is about 1.3 ms, and the fall time is about 300 μs. The rise time of the RM257 PSLC shutter is about 1.1 ms, and the fall time is about 800 μs.
3.3 Monomer concentration effect
The concentration of the monomer plays an important role in determining the domain size, driving voltage and response time of PSLC scattering shutters. By increasing the monomer concentration, the interaction between the polymer network and LC molecules are greatly enhanced, and a stronger network formed. Thus the driving voltage increases and the free relaxation time becomes faster . Since the RM257 PSLC scattering shutter has better performance over the TMPTA shutter, we tried to optimize its electro-optical properties with different RM concentrations. We find that the driving voltage of the PSLC films (9 μm) increases by increasing the RM257 concentration, as shown in Fig. 5(a). As illustrated in Fig. 5(b) the rise time (free relaxation time) decreases while the fall time increases as the RM257 concentration increases. It should be noted that, the saturation voltage of the 3 wt% RM257 film is quite low only about 12V,but the response time is too slow. The PSLC scattering shutter with 5 wt% RM257 has a saturation voltage of about 20 Vrms and a total response time of about 1.7 ms (rise time 1.1 ms and fall time 0.6 ms). So in consideration of both low voltage and fast response time the PSLC with 5 wt% RM257 is a good choice for our system. We also fabricated 5 wt% RM257 PSLC scattering films with the different cell gaps. Figure 6 shows the V-T curves of the PSLC films with 6 μm and 9 μm cell gaps, respectively. The 6 μm PSLC scattering shutter has a lower saturation voltage about 15 Vrms, since as the cell gap gets smaller the electric field E becomes stronger. The total response time is about 2.2 ms (rise time 1.6 ms, fall time 0.6 ms) which is close to that of the 9 μm cell.
4. Prototype system design
Based on the system scheme in Fig. 1, we implemented a proof-of-concept two-plane color optical see-through HMD prototype using off-the-shelf optical components. To obtain color images, here we employed a commercial color projector to project series of pictures onto the PSLC scattering shutters. The PSLC scattering shutters were operated at room temperature and worked well for wavelength from 400 nm to 700 nm. The projector had been originally designed to display images at a certain depth with a limited depth of field. The images before or after this depth plane could not be observed clearly. To ensure clear pictures displayed on each PSLC film at different physical depth, we designed an additional optical system to collimate the projected light as shown in Fig. 7(a). The system consists of three lenses (l1, l2 and l3) and a pinhole. The lens l1 forms a real image of the original picture at the front focal plane of l2, so that the rays of a signal pixel become a collimated light beam after passing through l2. A pinhole is placed at the back focal plane of l2, where the beam diameter becomes the smallest. Since the pinhole location is also the front focal plane of the lens l3, the output image light becomes quasi collimated with a single angular spectrum, and thus provides a large depth of field. Then the 2D images on each PSLC shutter will always be sharp and in the same size, and therefore the images displayed at different depths viewed through the combiner will have the same FOV.
Utilizing the quasi-collimated light source design and the PSLC films with 5 wt% RM257 concentration, as shown in Fig. 1 (b) and (c), we demonstrated a realistic augmentation of two color virtual words “SJTU” and “2017” with two real objects. A digital camera was placed at the back focal plane of the ocular lens to capture the color image. Two real optical components were placed 20 cm and 80 cm from the digital camera, respectively. The projector was operated at 60 Hz, so the two-plane system had a refresh rate of 30 Hz. Figure 8 shows the pictures captured in our experiment. When focusing the digital camera at 20 cm, both the near object and the green virtual word “2017” can be clearly observed, while the far object and the blue virtual word “SJTU” are strongly blurred. When focusing the digital camera at 80 cm, the blue virtual word ‘SJTU’ and the far object are clearly observed, while the green virtual word “2017” and the near object are strongly blurred. Visualization 1 shows the captured video of the augmented scene, when changing the focusing distance of the camera. The system presents the depth information accurately which solves the accommodation-vergence conflict problem.
To implement more depth planes, a high speed (120 Hz) digital micromirror device (DMD) was used as an alternative projector. And a four-plane image monocolor display was implemented for AR applications with a refresh rate of 30 Hz. Four letters “S”, “J”, “T” and “U” were displayed at 20 cm, 40 cm, 80 cm and 500 cm from the camera, respectively. Because of the limited space only three objects were placed at 20 cm, 40 cm, and 80 cm from the camera.
The captured images when the camera was focused at the different letters are shown in Fig. 9. With better driver circuit design, we could display more planes for our system to achieve more continuous 3D scenes. With 6 planes ranging from 33 cm to infinity, we could achieve a depth interval (z-axis resolution) of 0.6 diopters, which is adequate to achieve retinal image quality comparable to the effect of typical depth of field of the human eye, according to Dr. Hong Hua’s research [8, 9]. And if more planes are displayed in the system, by using the reverse mode PSLC scattering shutters, our system will offer lower power consumption than the normal mode PSLC scattering shutter system. Because of the telecentricity system in object space is used in our display, we can have the same FOV and angular resolution for different image planes . To achieve better image qualities for the virtual objects at different depths, the ocular lens need to be further optimized.
In this paper, we have proposed an optical see-through multi-plane volumetric head mounted display design for augmented reality applications with reverse mode PSLC scattering shutters. The electro-optic properties of the PSLC scattering shutters with different monomer types, monomer concentrations and cell gaps are investigated in the paper. The polarization insensitive reverse mode PSLC scattering shutter with 5 wt% RM257 offers a low saturation voltage about 20 Vrms, a high contrast ratio 92%/5% and also a fast response time within 2 ms. Based on the reverse mode PSLC films, we implemented a proof-of-concept two-plane color prototype and a four-plane monocolor prototype at a refresh rate of 30 Hz. The prototypes could provide correct depth information, and offers a solution to the accommodation-vergence conflict problem in the conventional stereoscopic 3D displays. What’s more, by using the reverse mode PSLC scattering shutters, the system offers lower power consumption than using the normal mode PSLC scattering shutters if more than 2 depth planes are displayed.
National Key Research and Development Program of China (2017YFB1002902); National Natural Science Foundation of China (61727808, 61405114).
References and links
1. O. Cakmakci and J. Rolland, “Head-worn displays: a review,” J. Disp. Technol. 2(3), 199–216 (2006). [CrossRef]
2. J. Carmigniani, B. Furht, M. Anisetti, P. Ceravolo, E. Damiani, and M. Ivkovic, “Augmented reality technologies, systems and applications,” Multimedia Tools Appl. 51(1), 341–377 (2011). [CrossRef]
5. G. Li, D. Lee, Y. Jeong, J. Cho, and B. Lee, “Holographic display for see-through augmented reality using mirror-lens holographic optical element,” Opt. Lett. 41(11), 2486–2489 (2016). [CrossRef] [PubMed]
6. K. Wakunami, P.-Y. Hsieh, R. Oi, T. Senoh, H. Sasaki, Y. Ichihashi, M. Okui, Y.-P. Huang, and K. Yamamoto, “Projection-type see-through holographic three-dimensional display,” Nat. Commun. 7, 12954 (2016). [CrossRef] [PubMed]
8. X. Hu and H. Hua, “Design and assessment of a depth-fused multi-focal-plane display prototype,” J. Disp. Technol. 10(4), 308–316 (2014). [CrossRef]
10. H.-S. Chen, Y.-J. Wang, P.-J. Chen, and Y.-H. Lin, “Electrically adjustable location of a projected image in augmented reality via a liquid-crystal lens,” Opt. Express 23(22), 28154–28162 (2015). [CrossRef] [PubMed]
11. S. Liu, H. Hua, and D. Cheng, “A novel prototype for an optical see-through head-mounted display with addressable focus cues,” IEEE Trans. Vis. Comput. Graph. 16(3), 381–393 (2010). [CrossRef] [PubMed]
12. W. Song, Y. Wang, D. Cheng, and Y. Liu, “Light field head-mounted display with correct focus cue using micro structure array,” Chin. Opt. Lett. 12(6), 060010 (2014). [CrossRef]
13. S.-K. Kim, E.-H. Kim, and D.-W. Kim, “Full parallax multifocus three-dimensional display using a slanted light source array,” Opt. Eng. 50(11), 114001 (2011). [CrossRef]
14. D. Teng, L. Liu, and B. Wang, “Super multi-view three-dimensional display through spatial-spectrum time-multiplexing of planar aligned OLED microdisplays,” Opt. Express 22(25), 31448–31457 (2014). [CrossRef] [PubMed]
16. D. Dunn, C. Tippets, K. Torell, P. Kellnhofer, K. Akşit, P. Didyk, K. Myszkowski, D. Luebke, and H. Fuchs, “Wide Field Of View Varifocal Near-Eye Display Using See-Through Deformable Membrane Mirrors,” IEEE Trans. Vis. Comput. Graph. 23(4), 1322–1331 (2017). [CrossRef] [PubMed]
19. Y.-H. Lee, G. Tan, Y. Weng, and S.-T. Wu, “Switchable Lens based on Cycloidal Diffractive Waveplate for AR and VR Applications,” SID Symp. Digest48(1), 1061–1064 (2017). [CrossRef]
20. Y.-H. Lee, H. Chen, R. Martinez, Y. Sun, S. Pang, and S.-T. Wu, “Multi-image Plane Display based on Polymer-stabilized Cholesteric Texture,” SID Symp. Digest48(1), 760–762 (2017). [CrossRef]
21. G. D. Love, D. M. Hoffman, P. J. Hands, J. Gao, A. K. Kirby, and M. S. Banks, “High-speed switchable lens enables the development of a volumetric stereoscopic display,” Opt. Express 17(18), 15716–15725 (2009). [CrossRef] [PubMed]
22. S. Liu, Y. Li, P. Zhou, X. Li, N. Rong, S. Huang, W. Lu, and Y. Su, “A multi-plane optical see-through head mounted display design for augmented reality applications,” J. Soc. Inf. Disp. 24(4), 246–251 (2016). [CrossRef]
23. H. Ren, Y.-H. Lin, Y.-H. Fan, and S.-T. Wu, “Polarization-independent phase modulation using a polymer-dispersed liquid crystal,” Appl. Phys. Lett. 86(14), 141110 (2005). [CrossRef]
24. H. Ren and S.-T. Wu, “Reflective reversed-mode polymer stabilized cholesteric texture light switches,” J. Appl. Phys. 92(2), 797–800 (2002). [CrossRef]
25. D.-K. Yang, Fundamentals of Liquid Crystal Devices (John Wiley & Sons, 2014).
26. J. Sun and S.-T. Wu, “Recent advances in polymer network liquid crystal spatial light modulators,” J. Polym. Sci., Part B: Polym. Phys. 52(3), 183–192 (2014). [CrossRef]
27. F. Du, S. Gauza, and S.-T. Wu, “Influence of curing temperature and high birefringence on the properties of polymerstabilized liquid crystals,” Opt. Express 11(22), 2891–2896 (2003). [CrossRef] [PubMed]
28. I. Dierking, “Polymer network–stabilized liquid crystals,” Adv. Mater. 12(3), 167–181 (2000). [CrossRef]