This paper presents a wide angle holographic display system with extended viewing angle in both horizontal and vertical directions. The display is constructed from six spatial light modulators (SLM) arranged on a circle and an additional SLM used for spatiotemporal multiplexing and a viewing angle extension in two perpendicular directions. The additional SLM, that is synchronized with the SLMs on the circle is placed in the image space. This method increases effective space bandwidth product of display system data from 12.4 to 50 megapixels. The software solution based on three Nvidia graphic cards is developed and implemented in order to achieve fast and synchronized displaying. The experiments presented for both synthetic and real 3D data prove the possibility to view binocularly having good quality images reconstructed in full FoV of the display.
©2012 Optical Society of America
The three-dimensional (3D) display shall deliver high quality optical replica of a moving object or scene that can be viewed by an observer from different directions. These postulates can be realized by the holographic technique, which works with a complex optical wave such as one generated in a real world and therefore correctly reconstructs 3D object wavefronts [1, 2]. Recently liquid crystal based Spatial Light Modulators (SLM) have been recognized as the most feasible devices for designing holographic displays [3–8]. The most accurate reconstruction and highest diffraction efficiency are delivered by SLMs based on liquid crystal on silicon .
Despite of numerous advantageous the holographic display technique requires signals with enormous space bandwidth product (SBP), which is very difficult to obtain in practice and therefore the reconstructed 3D images may not be observed stereoscopically . Current available SLMs have insufficient value of SBP (small active area, large pixel size) and therefore holographic displays based on SLMs produce beams that are small in size and propagate in only narrow angular confinement. In order to overcome this problem several designs of holographic displays with focus on an increased SBP have been reported [11–17]. The most natural way to increase the SBP in holographic displays is to use multiple SLMs. In literature there are two main configurations proposed: with flat and circular configurations. The flat configuration has its limitation for extension of viewing angle for commercially available modulators . Therefore, the circular configuration is preferred.
In this paper, we present the research related to a holographic display with multiple SLMs in circular configuration , where multiple object beams meet in an image space and are extending the viewing angle of holographic image (Fig. 1 ). Unfortunately it is not possible to align the two SLMs pixel to pixel without gaps. SLM has a mounting component around the active area what results in pixel circular distribution with gaps. The display system with gaps is unable to reconstruct a continuous object field and this affects the quality of visual perception. The gaps in the pixel circular arrangement result in a modulation of an image intensity with lower resolution of the obtained views . The gaps can be removed using a beam splitting plate . However for this setup the viewing angle is limited by angular dimension of the pixels arrangement. In addition this configuration cannot provide easy way of extending viewing angle in both horizontal and vertical directions.
In this paper we present the spatiotemporal multiplexing method which extends the viewing angle of holographic display system (Fig. 1) in both directions. The method relies on use of an additional SLM placed in an image space of the display. The additional SLM is synchronized with SLMs on the circle and changes propagation direction of holographic beams generated at the image plane. This method closes gaps in the circular pixel arrangement (horizontal (H) extension of viewing angle) and produces an additional circle of SLMs located on top of the former one (vertical (V) extension of viewing angle). As a result of spatiotemporal multiplexing, in display system an image is formed by the array of 2(V)x12(H) SLMs. This solution extends effectively 4 times the number of SLMs (extension of spatial SBP) on the circle, reducing the display frequency by factor of 4 (reduction of temporal SBP).
This paper is organized as follows. In Section 2 we present the developed spatiotemporal multiplexing method: its concept and theory. Next Sections are devoted to the implementation issues of the method. Section 3 presents experimental setup of the display, while Section 4 discuses the software implementation of spatiotemporal multiplexing for fast and synchronized displaying of holographic data is discussed. Finally, in Section 5 we present the experimental data showing the holographic images obtained in the built display system.
2. Spatiotemporal multiplexing method
2.1. Concept of the method
The idea of the spatiotemporal multiplexing method is applied to the holographic display using two reflective SLMs on a circle is shown in Fig. 2(a) . The angular separation between the SLMs 1 and 2 is chosen to get fill factor of FF = αSLM/α0 = 0.5. Let first discuss the field u1 that is produced by the SLM 1 and subsequently modulated by the additional SLM M. The SLM M is modulated with linear phase, which changes the direction of propagation of the field u1 and the resulted u1M is a copy of u1 as originated from different location. In the example illustrated in Fig. 2 the SLM M is modulated with the plane wave of spatial frequency A at the time t, spatial frequency is selected so the modulated field u1M(A)(t) is equivalent to the one originating from the SLM 1A (Fig. 2(b)). The SLM 1A has center at the boundary of the SLM 1. The SLMs are designed for the dynamic optical field modulation with a certain refresh rate. Therefore at the subsequent time step t + Δt of the spatiotemporal multiplexing method the field u1(t + Δt) is modulated with conjugated plane wave (spatial frequency B) via SLM M. As a result the field u1M(B)(t + Δt) is obtained as would be reproduced by the virtual SLM 1B (Fig. 2(b)). The SLM 1 is synchronized with the SLM M and a single temporal image frame is composed from two sub frames produced in two time steps (Fig. 2(b)). At the time step 1 the field u1M(A)(t) is generated and at the step 2 the field u1M(B)(t + Δt). The combined fields represent a signal of angular size αS, that is twice the size of u1. This methodology can be repeated for SLM2. SLM2 is synchronized with M and the signals u2M(A,B) generated by SLM 2 are added to the single frame at corresponding time steps (sub frames) extending viewing angle to 2αS. In this way the angular field of view of the display may be extended significantly, with limitation determined by refresh rate. The spatiotemporal multiplexing method implemented in the paper requires refresh rate to be high enough to compose a single image frame from 2 subframes (in the case of horizontal extension) and from 4 subframes (for both horizontal and vertical extension). This single frame is delivered jointly by the SLMs on circle and the SLM M.
2.2. Theory of spatiotemporal multiplexing method
The concept of spatiotemporal multiplexing relies on the modulation of holographic field at the image plane with plane wave so virtual position of display SLM is changed. Let consider generation of the exemplary field u1M(A) that is shown in Fig. 3 . At the image plane X2 the SLM M changes a phase of the reconstructed wave:
The selection of a proper modulation frequency for holographic display system and 1D multiplexing method is presented in Fig. 3 using Wigner Distribution Function (WDF) representation at the display (a) and (b) the image plane. The fields u1M(A) and u1M(B) at the image plane shall cover the bandwidth 2Bx/λz that is twice a bandwidth of signal u1. This requires the modulation frequencies fxM(A) = Bx/2λz and fxM(B) = -Bx/2λz. Using this modulation at the plane X1 the signals cover spatial range of 2Bx and are modulated with corresponding carrier frequencies. Figure 3 also shows the signal generated by the SLM 2, and that the reconstructed signal bandwidth is continuously extended to 4Bx/λz.
The presented analysis assumed that the SLM M is parallel to the SLM 1. This condition cannot be satisfied for all of the SLMs. In this system the SLMs are arranged on a circle with constant angular separation and the fields un are generated with corresponding angular steps. However the angular separation of fields is governed by the grating equation:
3. Wide angle holographic display system with spatiotemporal multiplexing
The scheme of our setup is shown in Fig. 4(a) , where six phase only 1080P SLMs are arranged in a circle and illuminated with six parallel laser beams formed by three beam splitters and a set of mirrors . The light reflected from the SLMs that reproduce holographic data meets in reconstruction space where it is modulated and reflected from the SLM M. The six SLMs on the circle are synchronized with the SLM M. This is a practical realization of the concept of spatiotemporal multiplexing method introduced in Section 2. A low power laser beam is sufficient for naked eye observation and in this setup the total power after collimation was 3μW.
Since the active area of SLMs in horizontal dimension is 15,36 mm the virtual SLMs are shifted by half of this dimension. It gives continues view with no angular gaps in the observed image. The display is configured for the reconstruction distance zr = 300 mm and thus SLM M shall reply plane wave with spatial frequency fx = 48 mm−1. This gives an angular separation 5.87° between two adjacent real SLMs and results in approximately 35° viewing angle in which an observer can see a reconstructed image.
In the holographic display system the viewer can freely move within the horizontal viewing angle and can see an image of size 15.36 mm in X direction. This is a limitation of the introduced method, because the image cannot be larger than the SLM dimensions and linear Field of View (FoV) in X is FoV = λzrΔ−1 = 20 mm for zr = 300 mm and Δ = 8 µm. This value is only slightly larger then SLM dimension. However the border region of FoV have decreased intensity and resolution .
The size of the SLM in vertical direction is 8.6 mm, which is much smaller than the FoV. Therefore an observer cannot see entire reconstruction due to limited human eye pupil dimensions. This effect is equivalent to viewing through a keyhole in which case an observer has to move up and down to view different parts of an image. The size of the observed region i.e. a monocular FoV equals :Fig. 4(b)). The modulation frequency is fy = 27 mm−1. This extends image size to 8.6 mm. In Section 5 we show the experimental results with both multiplexing schemes. One scheme applies modulation in X direction (fm = [ ± 48mm−1, 0]) and the second in both (fm = [ ± 48mm−1, ± 27mm−1]). In the second method there are four modulations and one holographic image frame is composed from four temporal sub frames. Each sub frame is obtained as generated from different virtual SLM (Fig. 4(b)). 1080P SLM is characterized by 60 Hz refresh rate. Therefore maximum display frequency for two dimensional modulation is 15 Hz.
4 Software implementation of spatiotemporal multiplexing method
The spatiotemporal multiplexing method requires synchronization of six display SLMs with modulating SLM M. To reach this goal a special display software is developed that facilitates modulation with four (or N) temporal sub frames. In our system SLMs are driven by the host computer with three graphic cards with Nvidia GTX-580 GPU. The GPU 1 and the GPU 2 drives the display SLM's 1-3 and 4-6 respectively, and the third GPU 3 drives the SLM M (Fig. 4).
The display software consists of two modules - the data stream buffering module and the principal display module. The first one is executed on the host computer and the second one on the graphic card. The data flow in the developed display software is organized in the two data transmission lines based on asynchronic transition mechanism between input stream and memory of graphic cards (Fig. 5 ). The transmission lines are responsible for delivery of data for the GPU 1 and 2 . The buffering process for the GPU 1 and 2 is realized in two threads loading data from input steams. The data for SLM M is loaded to memory of the graphics card during software initialization process. All of the data is transmitted in frames and each of them consist of 4 sub frames.
The data flow in the transmission line starts with the data collection in cyclic input buffers and just after its collection an entire frame is sent to the corresponding graphic card. The process of sending frame to the graphic card uses the transmission mechanism based on Pixel Buffer Object (PBO). The main advantages of PBO are the fast data transmission to GPU memory using DMA and the asynchronic data transmission mechanism. At the same time the next frame of data is being collected from the input streams. This data collection mechanism permits to operate four buffers for each stream. The synchronization of two streams for data collection is guaranteed using global input buffers switching mechanism, where the switching signal is generated by the slowest thread.
In order to display loaded frames the GPU 1, 2 and 3 create three separate contents and connected display threads. Display threads replay sequentially corresponding sub frames and integrates them in the same time with the mutual synchronization. To synchronize display threads we use method based on callback release mechanism.
Display threads are executed in parallel in synchronization with data transfer threads sending data to PBOs. When the frame is loaded to PBO the thread sends message to synchronization thread, that a process is finished and waits for trigger signal. At the same time the callback signal is sent repeatedly to all display threads and the next frame is displayed when the threads of data loading and frame displaying are finished. The speed of the display software execution for four temporal sub frames is sufficient to obtain holographic display with frequency 12.5 frame/sec (individual sub frame is refreshed with frequency 50 Hz).
5. Experimental results
The functionality of the display system is presented experimentally, where specially designed synthetic holograms and digital holograms of real world objects are reconstructed. The synthetic holograms are designed using the method outlined in our previous work . As an input for the synthetic holograms we use 3D computer graphic scene which is composed from three chairs. The size of the reconstructed scene was selected to present key hole effect (Fig. 6 ) discussed in Section 3, and to show that it can be removed with the developed spatiotemporal multiplexing method (Fig. 7 ). The size of images was chosen to be larger than MFoV. We have put an effort to capture images to be equivalent to the naked eye views, for this we use digital camera with diaphragm set to 5 mm. In Fig. 6 there are three photos taken with 1° angular separation of camera. While taking these images the SLM M was turned off and acted as a mirror.
Figure 6 shows that when looking directly into SLMs an observer can see a small part of the reconstructed view only. Moreover full height of the object cannot be seen. To see other parts of the image viewer has to change the position. The situation is changed significantly when SLM M is modulated, in both X and Y directions with four temporal subframes of spatial frequencies fm = [ ± 48mm−1, ± 27mm−1]. The obtained views are presented in Fig. 7.
In Fig. 7 there are six photos taken with an angular step (6°) corresponding to next pair of the virtual SLMs. Digital camera's settings remained unchanged. The presented pictures show that using our method it is possible to increase the MFoV in vertical direction and obtain a wide and continues viewing angle in horizontal direction. Now an observer can view the entire reconstruction, which results in an easier and more comfortable binocular observation.
Finally, we present reconstructions for a real world object. The scene that is used is a model of ethanol molecule printed using a 3D printer (ZPrinter 650). The scene elements are printed using 3D printer (ZPrinter 650). Dimensions of the model are round about 7 mm high and 11 mm wide (Fig. 8(a) ), because reconstructed images had to fit to active area of the SLM. For this data series of digital holograms were taken with rotating object  and corrected with tilt correction algorithm . In the contrary to the simulated object we can capture holograms of real scene of different perspectives in horizontal direction only due to the arrangement of the capturing setup. Therefore during reconstruction process modulation in X direction is possible only and this allows to apply the multiplexing technique with two temporal subframes (fm = [ ± 48mm−1, 0]). This results in slightly cropped views in vertical direction, which are showed in Fig. 8(b). This figure presents two important issues, which we believe are novelties. The first is the generation of wide angle holographic reconstruction that can be fully viewed with a naked eye without elements like asymmetric diffusers spoiling an image three dimensionality. The second one shows the limitations of capture system with rotated object. The photos have been taken with the angular step 6° that corresponds to the next pairs of virtual SLMs. To observe full image in vertical direction the camera diaphragm was increased.
This paper presents a holographic display with six SLMs arranged on a circle (Fig. 1) and one SLM used for modulation. The basic design of holographic display features two important problems: small viewing angle and substantial gaps in angular pixel arrangement. Both problems are relevant to the quality and experience of watching reconstructed holographic objects. In this paper we present the spatiotemporal multiplexing method, that extends viewing angle in both horizontal and vertical directions. This method increases SBP of display from 12.4 to 50 megapixels. This is achieved on the cost of display frequency reduction. The method relies on the use of the additional SLM M located in the image space. The SLM M is synchronized with SLMs on the circle.
In the theoretical part of the paper we prove that in Fresnel region the spatiotemporal multiplexing method continuously extends viewing angle. The performed analysis assumes that the SLMs on circle are parallel to the SLM M, even though this cannot be met in practice, the calculated error is of the order of angular resolution of humans eye. Thus, this assumption is not affecting a given view.
For mutual synchronization of the SLMs on circle with the SLM M and fast display we have developed spatial display software for three Nvidia graphic cards that drive seven SLMs of the system. We have evaluated speed of execution and due to hardware limitation the individual sub frames are refreshed with frequency 50 Hz. This means that with the spatiotemporal multiplexing method the holographic data with four sub frames can be refreshed with frequency 12.5 frame/sec.
To illustrate advantages of the developed spatiotemporal method in our display system we show views of reconstructions obtained for two types of objects: the simulated and the real one. Experiments proved that with our method we are able to extend monocular FoV (dimensions of observed view) in both vertical and horizontal directions, which permits to observe images binocularly in full display FoV without head movement. Moreover, we are able to double the effective number of SLMs in both horizontal and in vertical directions, what corresponds to extension of viewing angle from 17.5° to 35° (H) and from 1.6° to 3.3° (V). The presented method gives display with fill factor FF = 1.
Practical holographic displays with viewer comfort require an enormous SBP. The spatiotemporal multiplexing method presented in the paper enables an substantial increase of SBP in display system by the factor of 4. What is important, our method can be improved with the technological developments. Faster refresh rates of SLMs will allow to change more holographic frames, therefore to increase the number of SLMs even more. The method potentially can be applied to other display configurations as well e.g. display systems with larger FoV , using either optical manipulations that exchange viewing angle with image dimensions or applying other beam steering devices e.g. Digital Light Deflector .
The research leading to these results has received funding from National Science Centre within the projects under agreement 2011/02/A/ST7/00365 and UMO-2011/01/N/ST7/06476. The authors would like to thank Bryan Hennelly from the Department of Computer Science, National University of Ireland for capturing digital holograms presented in Fig. 8.
References and links
1. F. Yaraş, H. Kang, and L. Onural, “State of the art in holographic displays: A survey,” J. Disp. Technol. 6(10), 443–454 (2010). [CrossRef]
2. T.-Ch. Poon Ed, Digital Holography and Three-Dimensional Display (Springer, Berlin, 2006).
3. H.-E. Kim, H.-S. Kim, K.-M. Jeong, and J.-H. Park, “Three-dimensional binocular holographic display using liquid crystal shutter,” J. Opt. Soc. Kor. 15(4), 345–351 (2011). [CrossRef]
5. D. P. Kelly, D. S. Monaghan, N. Pandey, T. Kozacki, A. Michałkiewicz, G. Finke, B. M. Hennelly, and M. Kujawinska, “Digital holographic capture and optoelectronic reconstruction for 3-D displays,” Int. J. Digit. Multimed. Broadcast. 2010, 759329 (2010). [CrossRef]
7. G. Finke, T. Kozacki, and M. Kujawiska, “Wide viewing angle holographic display with multi spatial light modulator array,” Proc. SPIE 7723, 77230A, 77230A-8 (2010). [CrossRef]
8. K. Choi, H. Kim, and B. Lee, “Full-color autostereoscopic 3D display system using color-dispersion-compensated synthetic phase holograms,” Opt. Express 12(21), 5229–5236 (2004). [CrossRef] [PubMed]
9. A. Michałkiewicz, M. Kujawińska, T. Kozacki, X. Wang, and P. J. Bos, “Holographic three-dimensional displays with liquid crystal on silicon spatial light modulator,” Proc. SPIE 5531, 85–94 (2004). [CrossRef]
11. J. Hahn, H. Kim, Y. Lim, G. Park, and B. Lee, “Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators,” Opt. Express 16(16), 12372–12386 (2008). [CrossRef] [PubMed]
12. F. Yaras, H. Kang, and L. Onural, “Multi-SLM holographic display system with planar configuration,” in Proceedings of IEEE Conference on (IEEE 2010), 1–4.
14. T. Kozacki, M. Kujawińska, G. Finke, W. Zaperty, and B. Hennelly, “Holographic capture and display systems in circular configurations,” J. Disp. Technol. 8(4), 225–232 (2012). [CrossRef]
15. T. Kozacki, M. Kujawińska, G. Finke, B. Hennelly, and N. Pandey, “Extended viewing angle holographic display system with tilted SLMs in a circular configuration,” Appl. Opt. 51(11), 1771–1780 (2012). [CrossRef] [PubMed]
16. M. Stanley, M. Smith, A. P. Smith, P. J. Watson, S. D. Coomber, C. D. Cameron, C. W. Slinger, and A. Wood, “3D electronic holography system using a 100mega-pixel spatial light modulator,” Proc. SPIE 5249, 297–308 (2004). [CrossRef]
17. R. Haussler, A. Schwerdtner, and N. Leister, “Large holographic display as an alternative to stereoscopic displays,” Proc. SPIE 6803, 68030M, 68030M-9 (2008). [CrossRef]