Open Access
Add to BibSonomyAbstract
Speed enhancement of integral imaging based incoherent Fourier hologram capture using a graphic processing unit is reported. Integral imaging based method enables exact hologram capture of real-existing three-dimensional objects under regular incoherent illumination. In our implementation, we apply parallel computation scheme using the graphic processing unit, accelerating the processing speed. Using enhanced speed of hologram capture, we also implement a pseudo real-time hologram capture and optical reconstruction system. The overall operation speed is measured to be 1 frame per second.
©2012 Optical Society of America
1. Introduction
Hologram capture under incoherent illumination has strong advantages over traditional coherent hologram capture in terms of system complexity and process readiness. Various techniques including multiple projection method [1,2], ray space method [3], integral imaging based method [4,5] have been proposed. Among these, integral imaging based method requires only a single lens array with a camera, enabling compact system configuration. In integral imaging based method, three-dimensional (3D) objects are captured through a lens array, producing an array of small perspectives called elemental images. The elemental images are then transformed to an array of orthographic view images. The orthographic view images are finally used to synthesize a hologram of the captured object. Although the principle of the integral imaging based hologram synthesis method has been verified, the operation speed has remained slow. The bottleneck of the operation speed is the hologram synthesis from the orthographic view images part. Since each orthographic view image accounts for a single point in the hologram, the process should be repeated numerous times, causing the speed reduction.
Recently speed enhancement of hologram generation has been reported. Y. Ichihashi et al. developed a calculation system specialized for hologram, achieving 10 frame per second(fps) for 100,000 points [6]. T. Shimobaba et al. reported 24 fps for 512 × 512 grid data using a graphic processing unit(GPU) [7]. These methods, however, calculate holograms using 3D voxel or mesh data without capturing real-existing objects. Fast calculation of the hologram from captured images of real-existing objects has been reported by K. Yamamoto et al. [8]. In this report, elemental image array captured using integral imaging technique is processed by a system consisting of 4 personal computers to generate hologram in real-time. However, the hologram generated in this report is an optical field of the 3D image reconstructed by integral imaging 3D display, not the optical field of the original 3D objects [4, 9].
In this paper, we report an implementation of the speed enhanced integral imaging based hologram capture and display system using a GPU. Parallel computing architecture of the GPU enables simultaneous computation of the hologram data from the orthographic view images, accelerating hologram synthesis process. The generated hologram is an optical field of not the integral imaging 3D display but the original 3D objects according to the principle of the integral imaging based 3D hologram synthesis [4]. The synthesized hologram is loaded in the spatial light modulator (SLM) and illuminated by a laser, reconstructing 3D image of the captured object. In our current implementation, 1 fps speed is achieved for whole process. To the authors’ best knowledge, this is the first report of speed acceleration of the integral imaging based hologram capture and display using a GPU for an optical field of original 3D object.
2. Fourier hologram synthesis using integral imaging
Figure 1(a) shows procedure of the Fourier hologram synthesis using integral imaging. The capturing system consists of a lens array and a camera. The lens array forms an array of the areal elemental images in its focal plane. These areal elemental images are captured by the camera and then transformed to an array of the orthographic view images. Note that each pixel in the elemental image represents a light ray passing through the principal point of the corresponding elemental lens with a specific angle. By collecting pixels at the same local position in every elemental image, it is possible to synthesize a view image of a specific view angle [4]. Since the corresponding light rays are parallel to each other, the synthesized image has orthographic projection geometry. By repeating this process for all pixels in the elemental images, the elemental image array can be transformed to an array of the orthographic view images. The final procedure of the Fourier hologram synthesis is to calculate the complex field value using the orthographic view images at each point in the hologram plane. Figure 1(b) shows this process in detail. Each orthographic view image is first multiplied by a linear phase function according to the projection angle and then integrated to yield the complex field value at corresponding position in the hologram plane. Repeating this process for all orthographic view images, the hologram can be synthesized.
The most time consuming part of the integral imaging based hologram synthesis is the hologram calculation part from the orthographic view image array. In the conventional implementation using a central processing unit(CPU), each orthographic view image is processed sequentially, causing much delay. Suppose Nx × Ny elemental images with Mx × My pixels per each are captured. By collecting corresponding pixel in every elemental image, Mx × My orthographic view images with Nx × Ny pixels per each are generated. Mx × My linear phase functions with Nx × Ny resolution are also prepared for the hologram synthesis. Finally in order to synthesize a hologram of Mx × My resolution, element-wise multiplication of two Nx × Ny matrices, i.e. phase function and orthographic view image, and summation of all elements in the product should be repeated Mx × My times.
In our implementation, this process is parallelized using a GPU. Each orthographic view image with an associated phase function solely accounts for a single point value in the hologram plane and does not affect other points in the Fourier hologram. Therefore, the hologram calculation using the orthographic view image can be processed independently at the same time. The parallel architecture of the GPU enables simultaneous calculations using multiple orthographic view images, reducing the processing time significantly. Figure 2 shows the processing flow of sequential and parallel processing.
3. Verification of the GPU based processing
The GPU based implementation is verified by comparing the resultant hologram with that of the conventional CPU based implementation. In the verification, Intel Core i7 3.4GHz processor and NVIDIA GeForce GTX 590 processor are used as a CPU and GPU, respectively. Figure 3(a) shows the elemental image capture system. The lens array has 1 mm elemental lens pitch and 3.3 mm focal length. Two letters ‘3′ and ‘D’ located at different distances from the lens array are used as a 3D object. Figure 3(b) shows the elemental images captured by the camera. 25 × 19 elemental images are captured with 32 × 32 pixel resolution for each. Based on obtained elemental images, 32 × 32 orthographic view images of 25 × 19 pixel resolution each are synthesized as shown in Fig. 3(c). In order to increase the pixel count of the hologram [4], the orthographic view images are then repeated by triple times except boundary images to form 94 × 94 images. Finally, a Fourier hologram of 94 × 94 resolution is synthesized from the orthographic view images in two implementations using a CPU and a GPU.
Fig. 3 Elemental image acquisition system for obtaining a holographic image and orthographic view image.
Figure 4 shows amplitude and phase distribution of the Fourier holograms synthesized using the CPU and GPU. As shown in the Fig. 4, there is no difference between two holograms.
Figure 5 shows numerical reconstructions at different distances of the hologram synthesized using the GPU. In Fig. 5, it is clearly seen that two objects '3′ and 'D' are focused at different reconstruction distances, revealing that the synthesized hologram pertains 3D information of the captured objects. This result, along with comparison shown in Fig. 4, confirms that the GPU based implementation synthesizes correct Fourier hologram as conventional CPU based implementation.
Figure 6 shows operation speed of CPU and GPU based implementations. When the resolution of the synthesized hologram is small, the difference is negligible. As the hologram resolution increases, however, the GPU based implementation outperforms the CPU based implementation significantly. This is because the larger hologram resolution requires larger number of the orthographic view images and thus the parallel computation of the GPU can be better exploited to accelerate the operation speed.
4. Implementation of pseudo real-time capture and display system
We have implemented a pseudo real-time hologram capture and display system using a GPU. Figure 7 shows the configuration of the implemented system. The 3D objects are captured using an integral imaging pickup system under incoherent illumination. The captured elemental images are processed by using a GPU to synthesize the hologram. The synthesized hologram is then fed to optical reconstruction system consisting of a laser illuminated SLM and a Fourier transform lens and finally reconstructed as a 3D image.
Figure 8 shows the implemented setup. In the integral imaging capture system shown in Fig. 8(a), the specifications of the lens array and the CCD are the same as described in section 3. The total resolution of the captured elemental images is 790 × 641. Two 'K' letters at different depths are used as a 3D image. In the optical reconstruction system setup shown in Fig. 8(b), a 532nm green laser is expanded and collimated to illuminate a reflection type SLM. The SLM has 1920 × 1080 resolution with 8μm pixel pitch and is used in an amplitude modulation mode. The reflected light from the SLM is Fourier transformed by a lens of 170mm focal length. The reconstructed 3D image is captured using a CCD.
Fig. 8 Implemented setup of pseudo real-time hologram incoherent capture and coherent reconstruction system.
Figure 9 shows the experimental result. The captured elemental images and generated orthographic view images are shown in Fig. 9(a) and 9(b), respectively. The amplitude and phase distribution of the final synthesized Fourier hologram are shown in Fig. 9(c).
Figure 10(a) shows the optical reconstruction result captured at different CCD camera locations. As shown in Fig. 10(a), left 'K' image and right 'K' image are focused at different camera positions, reflecting original alignment of the captured 3D object. Figure 10(b) demonstrates pseudo real-time operation of the system. Figure 10(b) shows optical hologram reconstruction result captured at a fixed camera position as the 3D object moves longitudinally. It can be observed that the captured image at the CCD changes instantaneously according to the object position, confirming pseudo real-time operation of the implemented system. In the current implementation, the operation speed from the object capturing to the optical reconstruction is measured to be 1 fps.
Fig. 10 Optical reconstruction result of the pseudo real-time capture and display system (Media 1 and Media 2).
Although experimental results in Fig. 10 show that the proposed method can capture and reconstruct the 3D hologram successfully in pseudo real-time, still there is large room for further enhancement. One is the enhancement of the hologram reconstruction quality. The reconstruction results shown in Fig. 10 exhibit significant noise around the reconstructed images. Main cause is thought to be calibration error in the integral imaging capture system. In the integral imaging capture system, each elemental image should be rectified and identified precisely so that the orthographic view images are generated without errors and thus the hologram can be synthesized correctly. In our current implementation, however, barrel distortion correction for whole elemental image array and accurate pixel registration for each elemental image are not included, lowering the optical reconstruction quality.
Another possible enhancement is the operation speed. In our current implementation, only the hologram synthesis part from the orthographic view image array is parallelized by using the GPU. By parallelizing whole algorithm, additional speed enhancement can be accomplished. Also, the current GPU programming code can be improved for better managing the system memory, data transfer and GPU kernels.
5. Conclusion
Speed acceleration of the integral imaging based 3D hologram capture using GPU is reported. In integral imaging based hologram capture method, each orthographic view image is multiplied with a linear phase function and integrated to yield a complex field value at a point in the hologram and this process is repeated for all points in the hologram. The proposed implementation parallelizes this process using a GPU, enhancing the operation speed by 3.4 times over CPU case for 280 × 280 resolution hologram. Using the enhanced speed, a pseudo real-time hologram capture and optical reconstruction system is also implemented. The experimental results show that our system captures 3D objects using integral imaging pickup system under incoherent illumination, synthesizes corresponding hologram, and optically reconstructs as holographic 3D images successfully in a speed of 1fps.
Acknowledgment
This research was partly supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (2011-27306). This research was also partly supported by the IT R&D program of MKE/KEIT. [KI001810035337, A development of interactive wide viewing zone SMV optics of 3D display]
References and links
1. N. T. Shaked and J. Rosen, “Modified Fresnel computer-generated hologram directly recorded by multiple-viewpoint projections,” Appl. Opt. 47(19), D21–D27 (2008). [CrossRef] [PubMed]
2. N. T. Shaked, B. Katz, and J. Rosen, “Review of three-dimensional holographic imaging by multiple-viewpoint-projection based methods,” Appl. Opt. 48(34), H120–H136 (2009). [CrossRef] [PubMed]
3. K. Wakunami and M. Yamaguchi, “Calculation for computer generated hologram using ray-sampling plane,” Opt. Express 19(10), 9086–9101 (2011). [CrossRef] [PubMed]
4. J.-H. Park, M.-S. Kim, G. Baasantseren, and N. Kim, “Fresnel and Fourier hologram generation using orthographic projection images,” Opt. Express 17(8), 6320–6334 (2009). [CrossRef] [PubMed]
5. N. Chen, J.-H. Park, and N. Kim, “Parameter analysis of integral Fourier hologram and its resolution enhancement,” Opt. Express 18(3), 2152–2167 (2010). [CrossRef] [PubMed]
6. Y. Ichihashi, H. Nakayama, T. Ito, N. Masuda, T. Shimobaba, A. Shiraki, and T. Sugie, “HORN-6 special-purpose clustered computing system for electroholography,” Opt. Express 17(16), 13895–13903 (2009). [CrossRef] [PubMed]
7. T. Shimobaba, Y. Sato, J. Miura, M. Takenouchi, and T. Ito, “Real-time digital holographic microscopy using the graphic processing unit,” Opt. Express 16(16), 11776–11781 (2008). [CrossRef] [PubMed]
8. K. Yamamoto, T. Mishina, R. Oi, T. Senoh, and T. Kurita, “Real-time color holography system for live scene using 4K2K video system,” Proc. SPIE 7619, 761906 (2010). [CrossRef]
9. T. Mishina, M. Okui, and F. Okano, “Calculation of holograms from elemental images captured by integral photography,” Appl. Opt. 45(17), 4026–4036 (2006). [CrossRef] [PubMed]
