Open Access
Add to BibSonomy
Abstract
Holographic displays have the feature to show images out of the plane of the device itself, which is especially favored for augmented reality (AR) applications where the images need to be merged with the real world. In existing cases of AR holographic display, a combiner is used to converge the light path of the display image and surrounding scene toward the viewer's eye. In this paper, the idea of combining the holographic device and the combiner has been proposed, resulting in a see-through holographic display. In order to maintain the see-through quality of the device, the concept of partial hologram has been introduced, which means only a part of the area on the device has the holographic fringe pattern while leaving the rest fully transparent. Experiment and theoretical investigation shows that an evenly yet randomly distributed partial hologram provides the best holographic image quality assuming a fixed percentage of the holographic area on the device. A passive computer generated hologram (CGH) with two phase levels has been designed and fabricated for the verification. With partial hologram sharing 25% of the whole area, the CGH exhibits 90.9% of total transmission and 72.2% of parallel transmission. The demonstration shows a high see-through quality while providing a clear holographic image.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
A holographic display has its phase/amplitude modulation pattern on the device and the desired image, no matter real or virtual, is formed at somewhere a distance from the device itself [1–3]. The distance between the image and the device can be controlled with the modulation pattern on the holographic display device [4,5]. The feature of generating images out of the plane of the display device with controllable distance matches well with the requirement of displays for AR. In many application scenarios of AR display, the image needs to be merged with the real world as seen by the viewer. One major factor influencing the sense of object immergence is the perceived longitudinal position relative to the real world [6,7]. If the environment scene changes, the perceived object distance might need to be changed in correspondence. As a consequence, holographic display has been widely investigated for AR application [8]. In all cases of those holographic AR displays, there is a combiner to converge the light path of the image and that of the surrounding scene toward the viewer's eye [9–11]. The combiner can be a holographic film [12,13], polarization beam splitter [14] or intensity beam splitter [15,16]. There becomes quite a space needed with these basic components for the construction of the overall holographic AR display. For an AR display which is often used in a mobile situation, the volume or form factor leaves a room for improvement.
In this paper, the idea of combining the holographic display device and the combiner has been proposed. The combination will result in a flat and compact device which can generate an image at desired distance away while allowing the viewer to see the surrounding scene through it. A phase modulation only holographic or diffractive element normally has a high overall transmittance. However, see-through quality is highly related to the parallel transmission, but not only overall transmission, of the device. Therefore, the concept of partial hologram has been introduced for the realization of the see-through holographic display, which means only a part of the area on the holographic display provides phase modulation pattern, leaving the rest fully transparent. Based on this concept, there is still an issue that how the partial hologram should be distributed to achieve the best image quality, assuming the holographic area occupies a fixed percentage of the overall area of the device.
In Section 2, the concept of partial hologram and the influence of its distribution pattern on the image quality are described. In Section 3, the optimization, fabrication and evaluation of a two phase level passive computer generated hologram is provided for illustrating the feasibility and effectiveness of the proposed architecture. Finally, Section 4 gives the conclusions.
2. Partial hologram and quality of reconstructed image
The architecture of the partial hologram is illustrated in Fig. 1, where the shaded area indicates a holographic area with fringe pattern on it. If the holographic area takes a fixed percentage, how the distribution of the holographic area influences the image quality becomes an important issue for the design of the see-through holographic display. According to scalar diffraction theory, the complex disturbance at holographic plane and that at the image plane have the relationship of being Fourier transform to each other [17,18]. From this viewpoint, the spatial position at the holographic plane represents the spatial frequency at the image plane. Therefore, even distribution of holographic area should be favored so that there will be no specific spatial frequency band missing in getting involved in the reconstruction of the image. There has also been theoretical simulation results to support this argument [19]. As a consequence, a periodic stripe distribution of holographic area shown in Fig. 1 has been adopted for further investigation by varying the pitch of the stripe pattern. The effect of one dimensional periodic distribution of holographic area can be extended to two dimensional cases.
The assessment is based on simulation by using iterative Fourier transform algorithm (IFTA) for the optimization of computer generated holograms [20]. Given the same target image, those computer generated holograms with 50% of holographic area but different pitch of stripe pattern are optimized with the same algorithm and the same criteria of cost function. The CGH for simulation has a pixel size of 6.4 µm and pixel number of 1920 × 1080, with 256 phase level. The wavelength is 0.532 µm and the target image distance is 90 cm away from the CGH. Four cases with different pitches of the holographic stripe pattern are designed and simulated for assessment, namely 2, 10, 20 and 60 pixels. Pitch of 10 pixels, for example, means 5 columns of pixel have phase modulation in optimization as a part of the holographic area and the other 5 columns of pixel always have no phase modulation as a part of transparent area. Gerchberg–Saxton (G.-S.) algorithm [21] is adopted for the optimization process and the cost function being used is root mean square error (RMSE).
Figure 2 shows the target image, and Fig. 3 shows the simulated reconstruction images for the stripe pattern CGH with different pitch. All the reconstructions show multiple images within the signal window except the case of 2 pixel pitch, although all their phase modulation over the holographic area have been optimized with an iterative process. The larger the pitch of the stripe holographic area, the smaller the gap between multiple images. For the case of 2 pixel pitch, the gap between multiple images becomes large enough so that the multiple images locate outside the signal window. Although the image quality could be acceptable for some specific conditions, the partial hologram with periodic stripe pattern of holographic area is still not adequate for display purposes in general.
Fig. 3. Reconstructed images of stripe pattern CGH with pitch of (a) 2 pixels (b) 10 pixels (c) 20 pixels (d) 60 pixels.
The phenomenon of multiple images can be explained theoretically with the Fourier transform relationship between the CGH plane and the target image plane. The stripe holographic area is mathematically represented by a rectangular function convolving with a comb function. At its Fourier plane, it becomes a sinc function multiplied by a comb function. To resolve this issue, a randomized stripe pattern has been proposed as illustrated in Fig. 4. Among 1920 columns of pixels on the CGH, 960 of them are randomly chosen as the holographic area, leaving the rest as the transparent area. The same optimization process is done to search for the best phase modulation over the holographic area, and the reconstructed image is shown in Fig. 5, with the RMSE value of 0.011797. It indicates that multiple images have been fully eliminated.
3. Binary CGH with randomly distributed holographic area
The concept of randomly distributed stripe holographic area can be directly extended to two dimensional equivalence, which becomes a randomly distributed square holographic area. A binary passive CGH has been chosen for proving the proposed concept. The CGH is a transmissive type and has only two phase levels, 0 and π respectively. The overall size is 4.5 cm × 4.5 cm and the pixel size is 2 µm × 2 µm. The pixel number becomes 22500 × 22500, among which only 25% of them are allocated as the holographic area, i.e., transparent area shares 75%. The approach to generate a randomized holographic area can be illustrated with Fig. 6. The blue square in the CGH contains 90 × 90 areas. Every blue square area is further divided into 4 areas as highlighted in the figure. Within each blue square, one among sub-area 2, 3, and 4 could be chosen as the holographic area, and the selection is based on a stochastic process. This holographic plate could be glued together with a see-through diffusive projection screen [23] in the future to make a hybrid type see-through display system for some specific applications, and sub-area 1 as shown with red square has been reserved for the diffusive area and kept as transparent area on the holographic plate.
The phase modulation over the holographic area of the CGH is then optimized with IFTA and the target image is shown in Fig. 7. There are two images at different positions to be reconstructed simultaneously, one is a virtual Pokémon image [24] 1 m behind the CGH with both a height and a width of 10 cm, and the other one a real disc image 10 cm in front of the CGH with a diameter of 1 cm. The illumination angle on the CGH is 7°.
The phase modulation pattern over the holographic area is optimized with IFTA, and the setting condition in the iteration is based on G.-S. algorithm, Because there are two images with a distance of 110 cm, the optimization has been done for each image individually and two phase modulation patterns are combined together based on layer-based method [25]. In realization of the CGH, only two phase level is available with the consideration of fabrication issues. However, for demonstrating the potential improvement on image quality with the advancement of technology, the case with 256 phase level has also been made for comparison. The RMSE in optimization all reaches steady value with 100 times of iteration. The RMSE of Pokémon and disc images with 2 and 256 phase levels are listed in Table 1. It indicates that more degree of freedom available on phase level do help to reduce RMSE. The simulated reconstruction images of Pokémon and disc with 2 and 256 phase level are shown in Fig. 8. It clearly shows that the ones with 256 phase level look brighter, hence higher diffraction efficiency, for both Pokémon and disc. The result also shows an overlap between the disc image and the zero order of directly transmitted light through a transparent area. As the illumination angle is only 7°, the zero order has not yet been fully separated from the disc image at only 10 cm away from the CGH.
Table 1. RMSE of Pokémon and disc image with 2 and 256 phase level in optimization
The integrated phase modulation pattern is then encoded as a binary microstructure. For the fabrication of the CGH, a silicon mold with corresponding negative microstructure is made with lithographic process. The final microstructure is made on a PET substrate by hot impression on a UV cured glue layer coated on the substrate. The CGH is shown in Fig. 9 highlighted with a yellow frame, which demonstrates a visually good see-through quality.
The image is reconstructed with an expanded laser beam of 0.532 µm wavelength. The experimental setup is shown in Fig. 10. Three printouts of Pokémon picture have been put at three longitudinal positions as the reference for verifying the focused positions of the camera, one at the Pokémon image (1 m behind the CGH), one at the disc image (10 cm in front of the CGH), and the other one in the middle (30 cm behind the CGH) where there is no image at all in focus. The pictures at these three focuses have been taken and are shown in Fig. 11. In taking the Pokémon image, as well as focusing at 30 cm behind the CGH, the camera is located at only 10 cm away from the CGH plate in order to catch the whole image through the CGH plate, while the distance becomes 35 cm from the CGH in capturing the disc image. Despite a white background, the images themselves are quite clear and locate at the positions as designed. The major reason for the low contrast of the images is insufficient diffraction efficiency resulting from only two phase levels available on the CGH for generating two images at different positions. In addition, the angle between zero order and first order in this case is only 7° and the reconstruction light directly passing through the transparent area can disturb the image if the image position is not far away enough from the CGH, as for the case of disc image. Nevertheless, those issues can be resolved with providing more phase level on the CGH and using structured light reconstruction illumination, and are not inherently unsolvable drawbacks of the proposed see-through holographic display.
For quantitative evaluation on the see-through quality, the transmissive property of the CGH has been measured with Nippon Denshoku NDH-700 haze meter, and the PET substrate is also measured for comparison. The result is listed in Table 2. With a transparent area sharing 75% of the total area, the CGH plate demonstrates a total transmission of 90.9% and a parallel transmission of 72.2%. The diffusive transmission is 19.2%, which corresponds to a haze of 21%.
Table 2. Measured transmission property of CGH and PET substrate
4. Conclusions
An architecture of a see-through holographic display with randomly distributed partial hologram has been proposed, which serves as the combination of image source and combiner for AR display application. Being a holographic device, the image position can be varied by changing the modulation on the device, and a transmissive type spatial light modulator would be required for dynamic application scenarios. A passive CGH with 75% of the transparent area has been made and demonstrated both clear holographic image and good see-through quality. The percentage of transparent area could be changed as required. The proposed architecture can provide a compact and flexible solution for AR display.
Funding
Ministry of Science and Technology, Taiwan (MOST 107-2221-E-009-085).
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References
1. F. Yaraş, H. Kang, and L. Onural, “State of the Art in Holographic Displays: A Survey,” J. Disp. Technol. 6(10), 443–454 (2010). [CrossRef]
2. Y. Lim, K. Hong, H. Kim, H. E. Kim, E. Y. Chang, S. Lee, T. Kim, J. Nam, H. G. Choo, J. Kim, and J. Hahn, “360-degree tabletop electronic holographic display,” Opt. Express 24(22), 24999–25009 (2016). [CrossRef]
3. F. Yaraş, H. Kang, and L. Onural, “Real-time phase-only color holographic video display system using LED illumination,” Appl. Opt. 48(34), H48–H53 (2009). [CrossRef]
4. A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Colour hologram projection with an SLM by exploiting its full phase modulation range,” Opt. Express 22(17), 20530–20541 (2014). [CrossRef]
5. H. C. Lin, N. Collings, M. S. Chen, and Y. H. Lin, “A holographic projection system with an electrically tuning and continuously adjustable optical zoom,” Opt. Express 20(25), 27222–27229 (2012). [CrossRef]
6. H. S. Chen, Y. J. Wang, P. J. Chen, and Y. H. Lin, “Electrically adjustable location of a projected image in augmented reality via a liquid-crystal lens,” Opt. Express 23(22), 28154–28162 (2015). [CrossRef]
7. Y. J. Wang and Y. H. Lin, “An optical system for augmented reality with electrically tunable optical zoom function and image registration exploiting liquid crystal lenses,” Opt. Express 27(15), 21163–21172 (2019). [CrossRef]
8. Z. He, X. Sui, G. Jin, and L. Cao, “Progress in virtual reality and augmented reality based on holographic display,” Appl. Opt. 58(5), A74–A81 (2019). [CrossRef]
9. J. Jeong, J. Lee, C. Yoo, S. Moon, B. Lee, and B. Lee, “Holographically customized optical combiner for eye-box extended near-eye display,” Opt. Express 27(26), 38006–38018 (2019). [CrossRef]
10. C. T. Draper, C. M. Bigler, M. S. Mann, K. Sarma, and P. A. Blanche, “Holographic waveguide head-up display with 2-D pupil expansion and longitudinal image magnification,” Appl. Opt. 58(5), A251–A257 (2019). [CrossRef]
11. C. M. Bigler, P. A. Blanche, and K. Sarma, “Holographic waveguide heads-up display for longitudinal image magnification and pupil expansion,” Appl. Opt. 57(9), 2007–2013 (2018). [CrossRef]
12. P. Zhou, Y. Li, S. Liu, and Y. Su, “Compact design for optical-see-through holographic displays employing holographic optical elements,” Opt. Express 26(18), 22866–22876 (2018). [CrossRef]
13. K. Wakunami, P. Y. Hsieh, R. Oi, T. Senoh, H. Sasaki, Y. Ichihashi, M. Okui, Y. P. Huang, and K. Yamamoto, “Projection-type see-through holographic three-dimensional display,” Nat. Commun. 7(1), 12954 (2016). [CrossRef]
14. T. Zhan, J. Xiong, J. Zou, and S. T. Wu, “Multifocal displays: review and prospect,” Photonics 1(1), 10 (2020). [CrossRef]
15. S. F. Lin and E. S. Kim, “Single SLM full-color holographic 3-D display based on sampling and selective frequency-filtering methods,” Opt. Express 25(10), 11389–11404 (2017). [CrossRef]
16. D. Wang, C. Liu, and Q. H. Wang, “Holographic zoom micro-projection system based on three spatial light modulators,” Opt. Express 27(6), 8048–8058 (2019). [CrossRef]
17. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1996).
18. D. G. Voelz, Computational Fourier Optics: A MATLAB Tutorial (SPIE, 2011).
19. S. H. Tseng, P. T. Lee, and C. H. Chen, “See-through holographic display for AR application,” International Conference on Display Technology (ICDT), Vol. 50, issue S1, pp. 355–357 (2019).
20. F. Wyrowski and O. Bryngdahl, “Iterative Fourier-transform algorithm applied to computer holography,” J. Opt. Soc. Am. A 5(7), 1058–1065 (1988). [CrossRef]
21. R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–246 (1972).
23. Z. Y. Su, C. H. Chen, P. H. Yao, W. H. Yang, W. C. Chao, J. J. Lin, and Daniel Wu, “See-through projection screen and display system,” Society for Information Display, P-144, Display Week 2015 (SID2015).
24. https://www.mightymedia.com.tw/
25. J. S. Chen and D. P. Chu, “Fast calculation of wave front amplitude propagation: a tool to analyze the 3D image on a hologram (Invited Paper),” Chin. Opt. Lett. 12(6), 60021–60024 (2014). [CrossRef]
26. ASTM International, “Standard Test Method for Haze and Luminous Transmittance of Transparent Plastics”, (2000).
