We present a parameter analysis of the integral Fourier hologram that is generated from multiple orthographic view images of a three-dimensional object. The maximum view angle, the lens array pitch, and the projection angle step are analyzed to reveal their effects on the maximum size of the reconstructed object and its maximum spatial frequency. With these analyses, we propose a lens array shift method to enhance the resolution of the reconstructed object from the Fourier hologram. The principles are verified by computational and optical experiment.
©2010 Optical Society of America
Holography has been considered the perfect technique for three-dimensional (3D) displays, since it can provide flawless 3D images with complete human depth cues. The complicated capture process of holograms, however, has been problematic. In order to simplify the hologram capture process, many hologram generation methods that use incoherent illumination have been reported . These methods use multiple view images that are taken under regular incoherent white illumination. Recently, a more advanced hologram synthesis method has been reported by J.-H. Park et al. . They generated a hologram from multiple orthographic view images of 3D objects based on integral imaging. In this method, a coherent optical system is not required. Moreover, it is possible to obtain the multiple orthographic projection images from a single, simple capture process. Therefore, the hologram capture process is much simplified. The manipulation of the 3D information of the objects is also easily achieved. However, the resolution of the hologram generated by this method is limited. Experimental verification has been performed only for simple objects of low detail. The exact theoretical analysis of the resolution of the hologram generated by this method has not been addressed yet.
Recently, many resolution enhancement methods for integral imaging have been proposed. S. Kishk et al. proposed an improved resolution method to detect and reconstruct 3D objects using time multiplexing computational integral imaging. The low-resolution limitation that is induced by the Nyquist upper limit has been overcome by using a non-stationary micro lens array. In the reconstruction, the researchers use an iterative back projection algorithm to reconstruct a high-resolution image . The use of synchronously moving micro-optics in the pickup and display of 3D integral imaging is proposed by J.-S. Jang et al. Also, a non-stationary micro lens array is used to overcome the Nyquist upper limit . L. Erdmann et al. used a scanning micro lens array in digital integral photography. They enhance the resolution of the element image by capturing multiple sets of the element images sequentially with a scanning micro lens array and a stationary pinhole array. The advantage of this system is that the resolution of the system is not limited by the number of camera pixels . Y.-T. Lim et al. proposed a lens array shift method that captures multiple sets of element images to get one combined set of element images to increase the spatial density of integral microscopy . These non-stationary lens array methods have demonstrated efficient resolution enhancement of integral imaging. However, they have not been applied to hologram applications.
In this paper, we present an analysis of the parameters that affect the resolution of the reconstruction of a Fourier hologram that is generated from multiple orthographic view images of a 3D object. Based on this analysis, we propose a method to enhance the resolution of the reconstruction by shifting the lens array.
In the analysis, we analyze both the maximum size of the reconstruction and its maximum spatial frequency. For the maximum size of the reconstruction, we found that the main factor is the orthographic projection angle interval. Too large a projection angle interval causes overlapping in the reconstruction space domain. For maximum spatial frequency, there are three factors, i.e. the capturing lens array pitch that determines the spatial sampling rate of the original 3D objects, the maximum orthographic projection angle, and the spatial frequency bandwidth of the object. The dominant factor is determined by the relationship among these three factors. Based on this analysis, we will prove that the lens array shift method can enhance the resolution of the reconstruction. The principles are verified both computationally and experimentally.
In section 2, the principle of the integral Fourier hologram is explained. Based on this principle, the analysis of the reconstructed object’s resolution is explained both in the space domain and the spatial frequency domain. The principle of the lens array shift method to enhance the resolution of the reconstruction is also explained. In section 3, the computational and optical experimental results are presented to verify the parameter analysis and the resolution enhancement by the lens array shift method.
2.1 Fourier hologram generation using multiple orthographic view images
Generating an integral Fourier hologram from multiple orthographic view images consists of two macro steps. The first step is the capturing of the 3D objects using a lens array and the generating of the orthographic images. In this step, the images of the objects are captured at the focal plane of the lens array. The captured image is comprised of element images that contain all of the perspective images of the 3D object, each of which is formed by the corresponding element lens of the lens array. The orthographic image is generated by collecting the pixels that have the same local position in each element image. Since only one pixel is extracted from each element image, the spatial sampling interval or the effective pixel pitch of the orthographic view image is given by the element lens pitch Δxp. Figure 1 shows a schematic of Fourier hologram generation using multiple orthographic view images. The two-dimensional lens array is depicted as one dimensional array in Fig. 1 for simplicity. In Fig. 1, all the blue points with the same projection angle θ=tan−1(s 1/l) ≈s 1/l are collected to form one orthographic image, where θ is the projection angle, l is the focal length of the lens array, and s 1 is the local distance of the element image pixel from the center of the element image in the x direction. As Fig. 1 shows, Δs is the pixel size of the element image, and θ max and Δθ are defined as the maximum projection angle and the angular interval of two adjacent projection lines, respectively. All the orthographic images are generated by applying this collection process to every pixel in the element images. The second step is generating a Fourier hologram from the orthographic images. To each orthographic image, the phase factor of a plane wave with the corresponding projection angle is multiplied. The product is integrated into a pixel in the Fourier hologram. The Fourier hologram is generated by repeating this process for all orthographic images .
With this Fourier hologram generating method, a one dimensional Fourier hologram of 3D objects can be written as ,Eq. (1) are given by Eq. (1) and Eq. (2), it can be seen that the hologram is discretized with Δu=MΔs=Δs2f/l=2fΔθ and its size 2Lu is given by 2Lu=2Ms max=4fθ max.
Note that, besides the maximum size and the spatial resolution of the reconstruction which will be discussed in the following sections, the axial resolution is also limited by the discrete nature of the captured orthographic images and the finite size of the generated hologram. The finite size of the generated hologram determines the numerical aperture (NA) which fundamentally restricts the depth of focus of the hologram. Since the NA of the hologram is given by NA=Lu/f=Ms max/f=2θ max in the integral Fourier hologram generation method using a lens array, the θ max is the main parameter determining the depth of focus of the generated hologram. On the other hand, the discrete nature of the orthographic image determines minimum axial shift of the object that causes barely detectable change in the captured orthographic images. From Fig. 1, it can be seen that, for a given orthographic image which corresponds to a projection angle θ=s/l, the barely detectable axial shift of the object is given by Δzs=Δxp/tanθ=Δxpl/s. Since all the orthographic images contribute to the synthesis of the holography, the minimum axial shift of the object that changes the synthesized holography is given by Δz=Δxp/tanθmax=Δxpl/smax. Hence from above two factors, the axial resolution can be roughly estimated.
2.2 Relations between optical fields at different planes
Generally, a 3D object can be written asEq. (3), the analysis presented in following sections can be applied to each slice of the object, enabling estimation of the reconstruction quality of the 3D object.
The Fourier hologram of the object slice O(x, y; z 0) is given by Eq. (4) isEq. (5) is the Fresnel propagation of the object slice over the distance z 0. Therefore we can see that the Fourier transform of the Fourier hologram H(u, v) is the object field at z=0 plane of the object slice at z=z0 plane.
From the above discussion, the process of 3D Fourier hologram generation and reconstruction can be represented by Fig. 2 . It can be divided into several steps. First, the object field O(x, y; z 0) is propagated to z=0 plane Оz =0(ξ, η). The integral Fourier hologram H(u, v) can be thought as a Fourier transform of Оz =0(ξ, η). In the reconstruction, by Fourier transform the hologram H(u, v) and the object field at z=0 plane О'z =0(ξ', η') can be obtained. Finally the object slice O'(x', y'; z 0) can be reconstructed by Fresnel propagating О'z =0(ξ', η') over distance z 0.
Suppose the object O(x, y; z 0) has lateral size 2Lx×2Ly and bandwidth 2Bx×2By. The object optical field at z=0 plane Оz =0(ξ, η) can be calculated by Fresnel propagating the object over distance z 0. The size of the object field at z=0 plane is given by 2Lξ=2Lx + 2Bx·λ·z 0 and the bandwidth is maintained at z=z 0 plane, i.e. 2Bξ=2Bx [8, 9].
2.3 Overlapping in the space domain of the reconstructed object
Figure 3 shows the space domain representation of the object optical field at z=z 0 and z=0 planes. The object field at z=0 plane is obtained by Fresnel propagating the object field at z=z 0 plane over distance z 0.
In the reconstruction, the object optical field at z=0, О'z =0(ξ', η') is obtained by taking Fourier transform of the hologram H(u, v). Since the hologram is discretized with Δu, by the sampling theory, the reconstruction of the object at z=0 plane exhibits repetition of the original field, as shown in Fig. 4 . The repetition period of the reconstructed objects at z=0 plane is given by λƒ/Δu=λ/2Δθ.
Figure 5 is the final reconstruction at z=z0 plane O'(x', y'; z 0). This is obtained by taking Fresnel transform to Oz =0'(ξ', η') over z 0. From Fig. 5, we can see that if we want to reconstruct the object without overlapping, we must satisfy inequality (6). Following inequality (6), the projection angle interval Δθ should be controlled to prevent overlapping of the reconstructed object.
2.4 Spatial frequency of the reconstructed object
The main factors determining the resolution of the reconstruction are element lens pitch Δxp and the maximum projection angle θ max. In the integral Fourier hologram generation process, orthographic view images that are synthesized by collecting the pixels from every element image are used. Since only one pixel is extracted from each element image, the spatial sampling interval of the orthographic view image is given by the element lens pitch Δxp. The 3D object is first sampled with Δxp in the orthographic view synthesis, and the orthographic view images are used in the hologram generation. Hence the reconstruction of the hologram reflects this sampling effect as a repetition of the original object field with 1/Δxp period in the spatial frequency domain. Another factor that determines the reconstruction resolution is the maximum projection angle θ max. As discussed in previous section 2.1, the half size of the hologram Lu is determined by the maximum projection angle θ max as Lu=2fθ max. According to the sampling theorem, the maximum frequency of the reconstructed object from the Fourier hologram is given by ,
Figure 7 shows the Fourier transform of the reconstructed field at z=0. From Fig. 7, it is obvious that in order to avoid aliasing in the reconstruction, the bandwidth of the object should be smaller than the sampling rate of the object, i.e.,
Figure 8 is the reconstruction of the object at z=z 0. The maximum frequency is limited by the maximum projection angle in the element image capturing process and the wavelength. If we want to reconstruct the object without high frequency loss, assuming that there is no aliasing, the cutoff frequency 2θ max/λ given by Eq. (7) must be larger than the bandwidth of the object,
If there is aliasing, however, we need to make the cutoff frequency as large as possible while rejecting the aliased frequency region. The condition is given by
2.5Principle of resolution enhanced Fourier hologram by using lens array shift method
We have seen that there are several limitations in the Fourier hologram generation from multiple orthographic images. One is the limitation of the projection angle, which is one of the main factors that affect the maximum frequency of the reconstructed object from the hologram. The other factor is the sampling rate of the 3D objects, which is determined by the element lens pitch of the lens array. Large element lens pitch limits the bandwidth of the reconstruction according to Eq. (10). In practical cases, the element lens pitch Δxp is not small enough to capture a 3D object of large bandwidth, making it the dominant factor limiting the resolution. It is possible to use a lens array with smaller lens pitch in order to increase the sampling rate. However, the lens pitch cannot be reduced arbitrarily. In addition, a small lens pitch decreases the element image size, making it hard to capture an image with sufficient pixel count. It also decreases the maximum projection angle unless the f-number is maintained.
The lens array shift method is one solution to this problem. We can use a lens array with proper lens pitch to maintain the maximum projection angle and sufficient element image pixel count, while increasing the sampling rate of the 3D object. In the lens array shift method, we capture four sets of element images and then combine the four sets of element images into one set of element images that has higher spatial density. Figure 9(a) shows the element images capturing process of the lens array shift method in the vertical direction. First, we set the lens array to a reference position and capture a set of element images. After this, the lens array is shifted along the vertical direction by half of the element lens pitch. With this lens array position, the other set of element images is captured. Then we shift the lens array along the horizontal direction over half of the lens pitch to the original reference lens array position and repeat the process to obtain another two sets of element images. Figure 9(b) shows the synthesis method using the four sets of element images. Four sets of element images are marked as EIij, where i is the index in the horizontal direction and j is the index in the vertical direction. The element images in each set of element images with the same index are gathered together as a group in the synthesized element image. The grouped four element images are located in the synthesized set according to their indexes as shown in Fig. 9(b). In the synthesized element images, the number of the element images is doubled, making the effective lens array pitch half. From this synthesized element images, integral Fourier hologram of higher resolution can be obtained.
3. Simulation and experiment results
3.1 Verification of the parameter analysis
We verified the parameter analysis computationally. Figure 10 is the object used in the simulation. The pixel count of the object is 200(H)×200(V), the size is 0.1m(H)×0.1m(V), and the bandwidth is 2Bx=2000. The wavelength we used in the computational Fourier hologram generation and reconstruction is 500μm. The distance of the object from the lens array is 0.2m.
Table 1 shows the parameters of the simulation to verify the maximum object size. According to all the other parameters and the previous analysis, the maximum Δθ that ensures no overlapping in the reconstructed object is 0.1432°. If Δθ is larger than this value, overlapping is expected to happen. Figure 11 is the space domain representation of the reconstructed object with the parameters in Table 1. Figure 11(a) is the representation of case (I) and Fig. 11(b) is the representation of case (II).
Figure 12(a) is the reconstruction with the parameters of case (I) in Table 1. The maximum allowable object size is nearly the same as the original object, thus there is no overlapping. Figure 12(b) is the reconstruction with the parameters of case (II) in Table 1. The Δθ is larger than the maximum Δθ. With this Δθ the maximum allowable object size is 7.9577cm, which is smaller than the original object size. In Fig. 12(b), as we expected, overlapping happens in the reconstruction.
Table 2 shows the simulation parameters to verify the maximum spatial frequency in the reconstruction. In cases (I) and (II), we verify the aliasing in the reconstruction, and case (III) and (IV) are used to verify the cutoff frequency. Here we maintained the small Δθ to reconstruct the object without any overlapping. From inequality (8), the maximum Δxp that avoids aliasing in the reconstruction is 0.5mm. If Δxp is larger than this value, there will be aliasing in the reconstruction. If the aliasing is very severe, the reconstruction will look like a collection of points. From inequality (9), the maximum projection angle θ max that is required to avoid high frequency loss is 14.324°. In cases (III) and (IV), the θ max is set to be smaller, so the cutoff frequency is smaller than the bandwidth of the object. In case (III), Δxp was set small to avoid the aliasing under the cutoff frequency. In case (IV), Δxp was increased to incur the aliasing under the cutoff frequency. Figures 13(a)-(d) are the spatial frequency domain representations of the reconstructions at z = 0 plane of each case in Table 2.
The results are shown in Fig. 14 . We can see that the resolution of Fig. 14(a) is the best because there is no aliasing and no frequency information loss. In Fig. 14(b), the aliasing makes the reconstructed image look like a collection of points. In Fig. 14(c), the high frequency component of the object is lost; hence the details of the object at the edges are degraded. In Fig. 14(d), the aliasing happens, and also the high frequency information is lost; thus the image quality is deteriorated.
3.2 Verification of the lens array shift method
We verified the resolution enhancement of the integral Fourier hologram by using the lens array shift method experimentally. We performed two experiments, one with two plane objects and the other one with a real 3D object.
In the experiment, the objects were captured by a lens array. The lens array consists of identical elemental lenses of 1mm × 1mm lens pitch and 3.3mm focal length. In the capturing process, we captured four sets of element images. We used a Sony DSLR α900 to capture the element images. We first captured one set of element images, and then shifted the lens array horizontally 0.5mm and captured the second set of element images. To the third and fourth sets of element images, the lens array was shifted vertically 0.5mm. We combined these four sets of element images into one set of element images with the proposed synthesis method as explained in section 2. Now we can regard this synthesized element images as one set of element images captured through a lens array with effective lens pitch of 0.5mm. From the original single set of element images and this synthesized set of the element images, the orthographic images were generated separately ; then, the Fourier Holograms were generated from the orthographic images. Finally the objects were reconstructed at different depths from these two integral Fourier holograms.
In the experiment, the mechanical movement of the lens array may cause position errors in the lens array shift. Since the object is sampled at each elemental lens position in the orthographic image capture process, the position error of the lens array shift leads to non-uniform sampling of the object, i.e. the effective sampling interval is not a constant Δxp/2 but fluctuates, which degrades final reconstruction quality by acting as broadband noise.
In our experiment, the minimum unit of the translate stage was 0.01 mm, which limits the maximum error of the lens array shifting under 0.01mm. This maximum error 0.01mm corresponds to sub-pixel shift, or 0.5 pixel shift, in terms of the captured elemental image pixel with our experimental setup (i.e. Δxp=1mm, pixel count of each element image =50). Consequently, the error caused by the mechanical movement of the lens array could be ignored in our experiment.
In the two plane objects experiment, the objects were located away from the lens array at 50mm for the left footprint and 65mm for the right one. Figure 15 shows the experiment setup. The total lateral size of the two objects was 5.8(H) ×7.5(V) cm. The pixel count of each element image was 50(H) ×50(V) pixels. Hence the pixel size of the element image Δs was calculated to be 1mm/50 = 20μm and the angular separation between the projection lines was Δθ=20μm/3.3mm=0.3472°. In the computational reconstruction, the wavelength was set to 300μm. With this Δθ, the maximum allowable object size is 2.475cm, which is much smaller than the real object size. Here we repeated each orthographic view image four times, making the effective Δθ be Δθ=0.3472°/4=0.0868°. Since in our experiment the apparent differences between neighboring orthographic view images were negligible, this simple repetition can be justified. In general, the intermediate view reconstruction (IVR) method can be used [11, 12]. With reduced Δθ = 0.0868°, the maximum allowable object size is now 9.9cm, which is larger than the object we used. Thus we can reconstruct the object without overlapping.
Figure 16(a) shows the conventional element images and orthographic images and Fig. 16(b) is the element images and orthographic images taken using the lens array shift method. With the lens array shift method, the number of the element images is doubled. Hence pixel count of each orthographic image is doubled.
Figure 17(a) shows the conventional reconstructed objects with different depths. The left footprint is reconstructed at a distance of 50mm and the right footprint is reconstructed at 65mm. The reconstructions are degraded because of the aliasing. Figure 17(b) is the reconstructed object done by using the lens array shift method. We can see that by using the lens array shift method, the resolution of the reconstruction is enhanced over the conventional one.
We also applied the proposed method to real 3D objects of continuous depth profile. Figure 18(a) is the experimental setup. All the experimental setup is the same as in the above experiment except for the object. Figure 18(b) is the two dimensional image of the real 3D object we used in the experiment. Figure 18(c) shows the distances to the lens array from different parts of the object. The size of the object is 4cm (H) ×4cm (V). In this experiment, the pixel count of the element image was 54(H) ×54(V); hence the pixel size of the element image was 1mm/54 = 18.52μm. The angular separation between the projection lines was Δθ=18.52μm/3.3mm=0.3216°. In the reconstruction, the wavelength was 300μm. With this Δθ, the maximum allowable object size is 2.673cm. Hence we also doubled the number of the orthographic view images by repeating them twice [11, 12]. Δθ was decreased to 0.1608°, making the maximum allowable object size 5.346cm.
Figure 19 shows the generated Fourier holograms. Figure 19(a) is the amplitude and phase of the conventional Fourier hologram, and Fig. 19(b) is the amplitude and phase of the Fourier hologram made using the lens array shift method. Figure 20 shows the reconstruction from the Fourier holograms in Fig. 19. Figure 20(a) is the conventional reconstruction; the object reconstruction is degraded because of the aliasing. Figure 20(b) is the reconstruction from the Fourier hologram made using the lens array shift method. Figure 20(c) is the comparison of the magnified reconstruction at z = 54mm in Fig. 20(a) and Fig. 20(b).It can be confirmed that the resolution is enhanced by the lens array shift method. It is also clearly seen that different parts of the object are reconstructed correctly at different depths; the left foot at about 48mm, the eyes at 51mm, the picture in the belly at about 54mm, and the legs clearly at about 63mm.
In this paper, we presented an analysis of the parameters that affect the resolution of the reconstruction of an integral Fourier hologram. It was found that the projection angle interval between neighboring orthographic view images determines the maximum size of the reconstruction field. It was also found that the maximum projection angle and the element lens pitch of the lens array determine the maximum spatial frequency of the reconstruction. Based on this analysis, the resolution enhancement using the lens array shifting method was proposed. The simulation and experimental results confirmed the validity of the analysis and the feasibility of the resolution enhancement by the lens array shift method.
This work was partly supported by a grant of the Korean Ministry of Education, Science and Technology (The Regional Core Research Program/Chungbuk BIT Research-Oriented University Consortium). This research was partly supported by National Research Foundation of Korea Grant funded by the Korean Government(2009-0069677)
References and links
2. J.-H. Park, M.-S. Kim, G. Baasantseren, and N. Kim, “Fresnel and Fourier hologram generation using orthographic projection images,” Opt. Express 17(8), 6320–6334 (2009), http://www.opticsinfobase.org/abstract.cfm?URI=oe-17-8-6320. [CrossRef] [PubMed]
3. S. Kishk and B. Javidi, “Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging,” Opt. Express 11(26), 3528–3541 (2003), http://www.opticsinfobase.org/abstract.cfm?URI=oe-11-26-3528. [CrossRef] [PubMed]
4. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. 27(5), 324–326 (2002). [CrossRef]
5. L. Erdmann and K. J. Gabriel, “High-resolution digital integral photography by use of a scanning microlens array,” Appl. Opt. 40(31), 5592–5599 (2001). [CrossRef]
6. Y.-T. Lim, J.-H. Park, K.-C. Kwon, and N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Express 17(21), 19253–19263 (2009), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-17-21-19253. [CrossRef]
7. J. W. Goodman, Introduction to Fourier Optics, 2nd ed. (McGraw-Hill, New York, 1996), Chap. 9, pp. 351–353.
8. A. Stern and B. Javidi, “Improved-resolution digital holography using the generalized sampling theorem for locally band-limited fields,” J. Opt. Soc. Am. A 23(5), 1227–1235 (2006). [CrossRef]
9. A. Stern and B. Javidi, “Sampling in the light of Wigner distribution,” J. Opt. Soc. Am. A 21(3), 360–366 (2004). [CrossRef]
10. J.-H. Park, J. Kim, and B. Lee, “Three-dimensional optical correlator using a sub-image array,” Opt. Express 13(13), 5116–5126 (2005), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-13-13-5116. [CrossRef] [PubMed]
11. L. Zhang, D. Wang, and A. Vincent, “Adaptive reconstruction of intermediate views from stereoscopic images,” IEEE Trans. Circ. Syst. Video Tech. 16(1), 102–113 (2006). [CrossRef]
12. J.-H. Park, G. Baasantseren, N. Kim, G. Park, J.-M. Kang, and B. Lee, “View image generation in perspective and orthographic projection geometry based on integral imaging,” Opt. Express 16(12), 8800–8813 (2008), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-16-12-8800. [CrossRef] [PubMed]