We demonstrate a tabletop holographic display system for simultaneously serving continuous parallax 3.2-inch 360-degree three-dimensional holographic image content to multiple observers at a 45-degree oblique viewing circumference. To achieve this, localized viewing windows are to be seamlessly generated on the 360-degree viewing circumference. In the proposed system, four synchronized high-speed digital micro-mirror displays are optically configured to comprise a single 2 by 2 multi-vision panel that enables size enlargement and time-division-multiplexing of holographic image content. Also, a specially designed optical image delivery sub-system that is composed of parabolic mirrors and an aspheric lens is designed as an essential part for achieving an enlarged 3.2-inch holographic image and a large 45-degree oblique viewing angle without visual distortion.
© 2016 Optical Society of America
Tabletop 360-degree three-dimensional (3D) display system is one of the most interesting and simultaneously challenging 3D display system configurations [1–3]. Its full 360-degree viewing zone range, consequential novel visual 3D experience, and necessary huge content data amount contrast it against conventional flat-panel based 3D displays with narrow viewing zone [4,5]. Because of its uniqueness and ultimacy, many researchers have actively investigated the possibility of the systematic realization of tabletop 360-degree 3D display.
In the research of tabletop 3D display, the use of electronic holographic display technology is encouraging [6–9] since, in principle, the holographic image synthesis provides continuous parallax, deep depth perception cue, and resolves the fundamental problems of non-holographic 3D display techniques such as accommodation-vergence conflict . A previous multiview-based tabletop displays have discretized views even with a support of hundreds of projectors , but the holographic display creates continuous parallax view within the viewing window. Hence, holographic tabletop display is a promising candidate for the tabletop 3D display.
In order to construct practical tabletop holographic 3D display, countermeasures for several technological issues should be devised. First, a large view volume is a requisite for commercial success, which is generally achieved by adopting spatial or temporal multiplexing techinques [7,11–13]. For this issue, ultrahigh resolution and high speed display panel accommodating full color R/G/B is an essential theme [14–20]. A previous tabletop system that used a digital micromirror device (DMD) showed a very limited small active display area , though it exploited high speed operation capability. The simple scaling of this configuration by the form of multiple array, i.e. spatial multiplexing, can be a candidate for extending the active display area, but it makes the optical system highly complicated. Regarding this, we should additionally consider the management of large aperture optical aberration inducing distortion of large size holographic image. The design of aberration-compensated optical system is issued. Second, highly off-axis viewing circumference should be created for practicality of tabletop vision system. The previous holographic tabletop display system has a technological limitation in this point since the decentered Fresnel lens cannot cover 45-degree off-axis viewing zone due to noisy stray light generation and light field distortion by aberration, thus the viewing angle of the previous system is restricted within maximum 15 degrees [2,6]. In practice, the system configuration of refractive optical elements gets bulky and multi-elemental for achieving such highly off-axis oblique imaging condition, which is difficult in alignment and weak in chromatic dispersion, while the system based on smooth reflective optical elements such as a parabolic mirror is more suitable for the same objective and has advantage of free chromatic dispersion. Third, considering the first and second points, we encounter special design and high degree of engineering of aspheric element to manage rotational symmetric large-aperture 45-degree off-axis imaging system.
In this paper, based on the above considerations, we design and demonstrate a tabletop holographic 3D display system that can serve 3.2-inch continuous parallax 360-degree holographic 3D image content to multiple observers at 45-degree oblique viewing circumference in real time. Synchronized four high-speed DMDs are used to comprise spatially-multiplexed 3.2-inch holographic panel and a specially designed aspheric optical imaging system is constructed for 45-degree off-axis holographic 3D imaging without aberration-induced visual distortion. The 2 by 2 DMD array is operated in the time-multiplexing mode to create dense 628 views on the viewing circumference at 20 frame per second, which is synchronized with a hollow-shaft motor.
2. Working principles and system design
The schematic regarding the design concept of the proposed tabletop holographic 3D display system is illustrated in Fig. 1. The aim of design is the construction of tabletop holographic 3D display system that allows an observer or multiple observers to be able to watch seamlessly-arranged 3.2-inch 360-degree electronic holographic 3D images while walking around the tabletop display. One of the key design factors is the viewing window size, which is closely related to the effective pixel size of the rotating image of virtual hologram. In operation-mode, observers watch the rotating tilted virtual hologram plane through the corresponding viewing window (VW), the rotation axis of which is on the center of the top surface of the display system. Here, the tilted virtual hologram plane is normal to optical axis, so it is not tilted to optical axis but to the rotational axis. The VWs are positioned on the viewing circumference, where the observer’s eyes are put to accept holographic image light field . The virtual hologram plane is a filtered optical image of DMD array as shown in Fig. 1.
In the proposed 360-degree tabletop electronic holographic display system, 2 by 2 DMD multi-vision is considered. A practical optical configuration for constructing 2 by 2 seamless DMD array is designed as shown in Fig. 2.
To realize spatial multiplexing technique, the active region of four DMDs are interknitted by aligning the beam of four DMDs and filtering the light wave of the 2 by 2 DMD array. Four DMDs, DMD1-4, are slightly decentered and their images are combined seamlessly by three beam splitters, BS1-3 to comprise a single large aperture panel. Four respective 4-f optical single sideband filter systems are installed to selectively pass the holographic signal and reject direct current (DC), conjugate noise, and higher-order diffraction . In the system, a single-side band filter is placed in the Fourier domain created by the 4-f optics, since the DMD modulates only the binary amplitude of the light field [21,22]. In detail, as two single-side band filters are applied in our system, they have the same size and the same position. DMD 1 and 2 share one, and DMD 3 and 4 share the other as is shown in Fig. 2. Their dimensions of the filtering domain, Whorizontal × Wvertical, are given by,Eq. (1a) and (1b).
In addition, the relationship between the wave field at the DMD plane, , and that at the filter plane, , can be given as follows [21,23].Eq. (2a)-(2c), M, N, comb and rect are the horizontal pixel number, the vertical pixel number, the comb function and the rectangular function, respectively.
The effective pixel size of the tilted hologram plane can be adjusted by the magnification factor of the optical relay system. In the system, L1, L2, L3, and L4 are the first Fourier transform lenses for DMD1-4, respectively, and after filtering with F1 and F2, the filtered signal is reconstructed at the ‘virtual hologram plane’ through the lens L5 and forms a seamless 2 by 2 DMD array here. At this stage, the effective pixel size at the virtual hologram can be effectively doubled of each DMD pixel number [21,22]. Here in the virtual hologram plane, the pixel size, pixel number, and panel size of the 2 by 2 DMD array are 13.68 µm × 13.68 µm, 1536 × 1536, and 1.17 inch, respectively. But those of the filtered virtual hologram were tuned to 27.36 µm × 13.68 µm, 768 × 1536, and 1.17 inch, where the horizontal pixel pitch is doubled by taking Nyquist sampling criteria into considerations . In other words, the horizontal pixel pitch at the virtual hologram plane after adopting filters is regarded as sampling interval rather than the physical pixel pitch at each DMD plane. In order to satisfy our design goal of 3.2-inch holographic image, it is necessary to magnify the virtual hologram plane with the magnification factor of 2.76 times. The rotational scanning parts should be followed up after the magnification. After magnification, the pixel size of the 3.2-inch holographic panel with effective pixel number of 768 × 1,536 is 75.2 µm × 37.6 µm. This pixel size specifies the rectangular VW size. The whole magnification procedure is described in Table 1.
In the system design, the exploitation of high-speed operation of DMD is intended to be maximized. The desired viewing position is the circular circumference band with 700 mm radius and 700 mm height above the tilted image of virtual hologram. Therefore, the VW is created 990 mm away from the center of the image of virtual hologram. In this study, the size of the VW is designed to be 14 mm × 7 mm on the band for 532 nm operating light wavelength. The VW size is given byEq. (1) is the physical pixel pitch of each DMD, s in Eq. (3) indicates the sampling interval. So, the demanded value of the sampling interval in the horizontal direction is 75.2 µm and that in the vertical direction is 37.6 µm. It means that 2.76 times magnification is necessary after virtual hologram with 1.17 inch diagonal. The necessary 2.76 times magnification is obtained by two steps. First, through the magnification optics with L6 and L7, the ratio of their focal lengths results in 1.67 times magnification because L6 and L7 have the focal length of 180 mm and 300 mm, respectively. In summary, lenses L1 through L6 have the focal length of 180 mm, and the lens, L7, has that of 300 mm. Second, the following off-axis scanning part composed of mirrors M3 and M4, aspherical field lens L8, and parabolic mirrors provides 1.65 times magnification. Here, the use of aspherical field lens is essential not only for the magnification but also for the formation of the VW.
In the off-axis configuration with the parabolic mirrors, the aspherical field lens is mandatory for electronic holographic application. With traditional parabolic mirrors as depicted in Fig. 3(a) , two mirrors are spaced by their focal length and they image the input plane at the bottom of the mirrors on the output plane at the top of the mirrors. Since the space between them is only their focal length, the diverging phase is produced on the output plane. Therefore, the traditional geometry does not properly form a viewing window because the diffraction angle of the hologram is too small to cover a large area. The spherically converging carrier wave is necessary to synthesize the VW as indicated in Fig. 2. Also it is significant for the off-axis distorted diverging optical field as shown in Fig. 3(a) to be compensated. In Fig. 3(b), the effect of the aspheric lens inserted between the parabolic mirrors in the proposed system is illustrated with ray tracing analysis.
3. System implementation
As is described in Fig. 2, the proposed system can be functionally divided into two parts, (i) 2 by 2 DMD combining and filtering part and (ii) magnifying and scanning part. The system prototype is presented in Fig. 4(a) with design drawings of the two parts. The design drawing and practical implementation of the magnification and scanning part are shown in Figs. 4(a) and 4(b), respectively. Figures 4(c) and 4(d) show the design drawing and practical implementation of the DMD combining and filtering part. Much effort was devoted to align the two optical parts to match the center of the rotating tilted image of virtual hologram to the cross point of the rotation axis and the top surface of the parabolic mirrors. The combining and filtering stage presents the feasibility of the scalable system, and the magnifying and scanning part renders the tabletop display comfortable for use by getting rid of any obstacles above it.
In the system layout, the lens L7, and mirrors M3 and M4, are grouped and this group is mechanically rotated under the parabolic mirror as indicated in Fig. 4(a). Therefore, the 3.2-inch holographic plane rotates as the viewing window revolves along the viewing circumference band. Here, the VW is set to 14 mm × 7 mm for the operating wavelength of 532 nm and the perimeter of the circumference band is 4.4 m.
With a single revolution, the VW rotates a complete 360-degree as clarified in . Taking account of this periodic rotation of the rectangular VW, we count the necessary number of VWs, at least 628 to compactly distribute the VW on the viewing circumstance band. Fortunately, each DMD (ViALUX GmbH, VIS-7000) has the operating speed of 22,727 Hz for binary images, so it is enough to broadcast more than 1,136 shots of 1,024 × 768 resolution images per one revolution at real time speed of 20 frame/sec. In the implemented system, 628 viewing windows are densely planted along a 700 mm radius circular circumference band with a 700 mm height from the top surface of the parabolic mirror which is shown in Fig. 4(a). Consequently, the system is implemented so that observers can see 3.2-inch holographic 3D contents by 20 frames per second, which is floating on the surface of parabolic mirror system. At the position of each viewer, the number of frames depends on the speed of the hollow motor, the maximum speed of which is 30 rps (revolution per second). In our experiment, the number of frame is 20 fps (frame per second). In addition, each DMD possesses internal random access memory (RAM) of 4GB (giga byte), and the amount of data loaded to each DMD is approximately 1.23GB per second which is given by 20 rps × 624 (number of viewing windows) × 1024 × 768 (pixel number of each DMD) ÷ 8 (conversion of bit to byte). In addition, since the size of the designed viewing window is 14 mm × 7 mm, either one or two VWs may cover viewers’ pupil in single location. However, while the hollow motor is rotating, the viewing windows rotate simultaneously. So, it is necessary to adopt the square part of the viewing window in order to satisfy the symmetric feature of 360 degree views. As a result, the square part out of the designed VWs, 7 mm × 7 mm, is adopted to generate 624 viewing windows. As a result, the number of VWs is 624 that can be seamlessly arranged on the viewing circumference.
In this system, a continuous-wave diode-pumped solid-state laser with a wavelength of 532 nm (Cobolt SambaTM, Cobolt AB) is used as a light source; its maximum output power is 300 mW. The light source is evenly divided into quarter by the use of a fiber-type beam divider (Thorlabs Inc.), and the fiber-coupled laser light is expanded uniformly after passing through a collimator (OZ Optics Ltd.). Four high-speed DMD modules (ViALUX GmbH, VIS-7000) are used as spatial light modulators (SLMs), and the lenses were designed and fabricated for this display. The DMD has a pixel pitch of 13.68 μm.
A Giant Mirage (Opti-Gone International, Inc.) is used as the parabolic mirrors, and a hollow-shaft direct drive motor was manufactured to our specifications by Justek Corp. on our demand. The motor is placed inside the scanner and rotates two folding mirrors (M3 and M4) and one aspheric lens (L8), as is shown in Fig. 4(a). The folding mirrors determine the propagation direction of the optical field from the scanner. The motor rotates at the speed of 20 revolutions per second, and more than 628 viewing windows can be formed during one turn. Consequently, a 39.5 Gbps, equivalent to 4.9 GB per second, data transfer rate is necessary for this system. Currently, the internal memory of the high-speed DMD modules is used and thus only a limited length of movie play is possible.
In addition, three kinds of lenses were designed for our system. Two achromatic lenses with different focal lengths are used for the Fourier transform. The lenses with a focal length of 180 mm, are used for the 4-f optics, and the lens with a focal length of 300 mm is used for magnification. The aspheric lens in the parabolic mirrors is designed to make the collimated incident light converge onto the viewing window after sequentially passing through the aspheric lens and the parabolic mirrors. Their prescriptions are listed in Table 2.
4. Experimental results
The observation of computer-generated holograms (CGHs) loaded in the system was tested. The CGH was designed using the point-cloud CGH synthesis method. The target 3D model and the CGH pattern were shown in Figs. 5(a) and 5(b), respectively. The total area of the CGH pattern is split into four sections that are seamlessly displayed on four DMDs. The object is modeled as a point cloud, and the size of the hologram is set to 1,536 × 1,536 points, the maximum size of the square that can be encoded by four DMDs. When the scanner rotates, the encoded hologram also rotates on the plane of the arrayed DMDs. Therefore, the object needs to be contained inside the circle within a diameter of 1,536 points, and only the square-shaped hologram is computed and encoded on the arrayed DMDs. In particular, CGH content was calculated to uniformly distribute light energy on the VW to sustain continuous parallax. The corresponding reconstructed image is shown in Fig. 5(c).
The hologram images captured by four cameras at four different positions, i.e. front, rear, right, and left indicated in Fig. 6(a) are respectively shown in Figs. 6(b)-(e). These are four sides of the wire frame of a human head with 90-degree intervals. Continuously-arranged VWs between the front view in Fig. 6(b) and the right view in Fig. 6(c) can be seen in Visualization 1 in Fig. 6. Since the proposed tabletop holographic display uses time-multiplexing technique of VWs, each CGH is assigned to a specific VW. Therefore, the occlusion between objects is naturally displayed. In the photos of CGHs, the occlusion effect such that the wire on the front is displayed and the wire at the back is screened by the front surface is confirmed.
As is seen in the experimental results, Fig. 5(c) and Figs. 6(b)-(d), unwanted issues such as aberration and brightness mismatch among four DMDs are observed. One issue is aberration mainly caused by parabolic mirrors, but it takes a lot of time to measure it at every location of VWs. Estimating and improving aberration in the proposed configuration is remained as our future work. Another issue is regarding the lower brightness of the certain DMD, the upper right one. The main difference is caused by the illumination optics, which is composed of total internal reflection prism, placed in front of each DMD. The uniformity is different, so the brightness differs from each other. In addition, though we used four 6-axis mechanical stages to align four DMDs, the alignment of each DMD launched on the 6-axis stage is not perfect. When we take a picture of generated holograms, it is hard to get the perfectly-aligned virtual image plane. The other issue is regarding the image quality of the reconstructed holograms. Since each DMD requires binary-type data, binarization of the calculated complex field causes the loss of information and mainly affects the image quality of reconstructed holograms. To solve this problem in using DMDs, the temporal multiplexing technique to reconstruct gray-level holograms can be applied by referring to the previously proposed work . However, in this work, as we focus on the realization of 360 degree holograms, we do not consider additional techniques. However, we are developing another version of table-top holographic display system by considering and trying to solve those inherent issues, and the renovated one is going to be presented in near future.
In conclusion, we have investigated technological innovation in the development of 360-degree holographic tabletop 3D display with highly oblique observation viewing circumference. As indicated in the introductory part, the issues of the enlargement of virtual hologram and the seamless continuous parallax holographic 3D image generation at the 45-degree off axis viewing circumference were addressed. This study has remained many research issues and one of them is the scale reduction or minimization of the system. The integration of red, green, and blue color components for full-color holographic display in the scale-downed tabletop system will be a very challenging topic. In addition, the active area of the parabolic mirror system is only less than 10% of the parabolic mirror area when considering that the diagonal size of the virtual hologram plane is 3.2 inch and the major-axis lengths of the parabolic mirrors are 22 inch. The waste of the space is a serious problem. Introduction of advanced freeform optic technology is considered as a prospected solution. Besides the system design issues, the CGH for DMD is binary CGH and it results in the degradation of the image quality severely. The binary CGH optimization problem is still an open problem in the CGH research field [24, 25]. Basically, the tabletop display generates 360-degree continuous horizontal parallax. However, the small vertical length of VW restricts the vertical viewing angle tightly. The enhancement of vertical viewing angle is also a pending issue in the development of holographic tabletop display.
GigaKOREA Project (GK16D0100, Development of Telecommunications Terminal with Digital Holographic Tabletop Display).
References and Links
3. A. Butler, O. Hilliges, S. Izadi, S. Hodges, D. Molyneaux, D. Kim, and D. Kong, “Vermeer: direct interaction with a 360° viewable 3D display,” Proc. ACM UIST, 569–576. (2011). [CrossRef]
5. D. Fattal, Z. Peng, T. Tran, S. Vo, M. Fiorentino, J. Brug, and R. G. Beausoleil, “A multi-directional backlight for a wide-angle, glasses-free three-dimensional display,” Nature 495(7441), 348–351 (2013). [CrossRef] [PubMed]
7. Y. Sando, D. Barada, and T. Yatagai, “Holographic 3D display observable for multiple simultaneous viewers from all horizontal directions by using a time division method,” Opt. Lett. 39(19), 5555–5557 (2014). [CrossRef] [PubMed]
8. T. Kakue, T. Nishitsuji, T. Kawashima, K. Suzuki, T. Shimobaba, and T. Ito, “Aerial projection of three-dimensional motion pictures by electro-holography and parabolic mirrors,” Sci. Rep. 5, 11750 (2015). [CrossRef] [PubMed]
9. P. St. Hilaire, “Electronic display system for computational holography,” Proc. SPIE 174, 1212–1220 (1990).
11. T. Kozacki, G. Finke, P. Garbat, W. Zaperty, and M. Kujawińska, “Wide angle holographic display system with spatiotemporal multiplexing,” Opt. Express 20(25), 27473–27481 (2012). [CrossRef] [PubMed]
12. D. Teng, L. Liu, Y. Zhang, Z. Pang, S. Chang, J. Zhang, and B. Wang, “Spatiotemporal multiplexing for holographic display with multiple planar aligned spatial-light-modulators,” Opt. Express 22(13), 15791–15803 (2014). [CrossRef] [PubMed]
14. A. W. Lohmann, R. G. Dorsch, D. Mendlovic, C. Ferreira, and Z. Zalevsky, “Space–bandwidth product of optical signals and systems,” J. Opt. Soc. Am. A 13(3), 470–473 (1996). [CrossRef]
15. K. Maeno, N. Fukaya, O. Nishikawa, K. Sato, and T. Honda, “Electro-holographic display using 15 mega pixels. LCD,” Proc. SPIE 2652, 15–23 (1996). [CrossRef]
16. M. Stanley, R. W. Bannister, C. D. Cameron, S. D. Coomber, I. G. Cresswell, J. R. Hughes, V. Hui, P. O. Jackson, K. A. Milham, R. J. Miller, D. A. Payne, J. Quarrel, D. C. Scattergood, A. P. Smith, M. A. G. Smith, D. L. Tipton, P. J. Watson, P. J. Webber, and C. W. Slinger, “100 mega-pixel computer generated holographic images from active tiling–a dynamic and scalable electro-optic modulator system,” Proc. SPIE 5005, 247–258 (2003). [CrossRef]
17. J. Hahn, H. Kim, Y. Lim, G. Park, and B. Lee, “Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators,” Opt. Express 16(16), 12372–12386 (2008). [CrossRef] [PubMed]
19. M. Kujawinska, T. Kozacki, C. Falldorf, T. Meeser, B. M. Hennelly, P. Garbat, W. Zaperty, M. Niemelä, G. Finke, M. Kowiel, and T. Naughton, “Multiwavefront digital holographic television,” Opt. Express 22(3), 2324–2336 (2014). [CrossRef] [PubMed]
20. H. Sasaki, K. Yamamoto, Y. Ichihashi, and T. Senoh, “Image size scalable full-parallax coloured three-dimensional video by electronic holography,” Sci. Rep. 4, 4000 (2014). [PubMed]
21. H. Kim, C.-Y. Hwang, K.-S. Kim, J. Roh, W. Moon, S. Kim, B.-R. Lee, S. Oh, and J. Hahn, “Anamorphic optical transformation of an amplitude spatial light modulator to a complex spatial light modulator with square pixels [invited],” Appl. Opt. 53(27), G139–G146 (2014). [CrossRef] [PubMed]
22. O. Bryngdahl and A. Lohmann, “Single-Sideband Holography,” J. Opt. Soc. Am. 58(5), 620–624 (1968). [CrossRef]
23. M. Park, B. Chae, H. Kim, J. Hahn, H. Kim, C. Park, K. Moon, and J. Kim, “Digital Holographic Display System with Large Screen Based on Viewing Window Movement for 3D Video Service,” ETRI J. 36(2), 232–241 (2014). [CrossRef]
25. Y. Im, H. Kim, and J. Hahn, “Iterative Fourier transform algorithm based on the segmentation of target image for a high-speed binary spatial light modulator,” J. Opt. Soc. Korea 19(2), 149–153 (2015). [CrossRef]