## Abstract

We propose to improve the depth of field of Integral Imaging systems by combining an array of phase masks with the traditional lenslet array. We show that obtained elemental images are sharp over a larger range than with a regular lenslet array. We further increase the quality of elemental images by a digital restauration. Computer simulations of pickup and reconstruction are presented.

© 2007 Optical Society of America

## 1. Introduction

Integral Imaging (II) is rising as one of the most promising and convenient ways to form three-dimensional images [1, 2]. It uses incoherent light so it does not suffer from the same speckle problem as holography. In addition, unlike stereoscopy, it does not cause visual fatigue. Still there are some challenges to overcome [3]–[13]. One of them is the short depth of field imposed by the lenslet arrays. To remedy this problem Jang et al. have presented in [3] a method of displaying the image throughout real and virtual image fields without introducing dynamic movements or additional devices. In [4] the same authors proposed to capture distant and large objects by using a curved pickup (and/or display) device. They have also proposed in [5] a time multiplexed integral-imaging method by the use of an array of lenslets with different focal lengths and aperture sizes. Separately, Hain et al. [6] enhanced the depth of field exploiting a similar idea by the use of diffractive optical elements, namely using an array of binary zone plates with different focal lengths; Jung et al. have proposed in [7] the use of different optical path lengths by using a polarization selective mirror pair or mirror barrier array; Martínez-Corral et al. have improved the depth of field of an II system by the use of an annular binary amplitude filter [8]. Martínez-Cuenca et al. have enhanced the depth of field by reducing the fill factor of each lenslet and by using an amplitude-modulated lenslet array and a deconvolution operation [9, 10]. In this contribution we propose to increase the depth of field of II systems by the use of an asymmetric phase mask. This phase mask has the inherent property of preserving the light gathering power [14]–[16]. In addition, when placed in front of each lenslet, it has the ability to preserve all the frequency content within its passband even for planes that are far away from the in-focus object plane. However this highly desirable characteristic is achieved at the expense of a deterioration of the visual image quality. Nonetheless since the elemental images can be captured through a CCD camera [17], then it is possible to perform a digital enhancement of the elemental images before performing the reconstruction stage either optically or digitally [18]–[20]. Computer simulations demonstrate the feasibility of our proposal.

## 2. Description of the problem

Integral Imaging is aimed at capturing and reproducing three-dimensional (3-D) views of objects. The II process therefore includes two steps: the pickup stage and the reconstruction stage. The ideal implementation of both stages is based on the use of a pinhole array. During pickup, each pinhole forms an image taken from a particular point of view by selecting a single ray from each point of the object. Each of these slightly different images is called elemental image. Now, during the reconstruction stage, the viewer observes the elemental images through the pinholes. The rays coming from different elemental images but corresponding to the same object point converge to the exact 3-D position of the original point. Therefore, the viewer has the illusion that the rays are emitted from this particular 3-D location. Since this process occurs for every point of the original object, the viewer actually sees a 3-D reconstruction of the object. Unfortunately, it is not possible to use a pinhole array in practice because the light gathering power would be very low. In order to avoid this problem, the pinhole array is generally substituted with a lenslet array.

The principle of II with a lenslet array remains the same as with a pinhole array. However problems arise from the fact that a lenslet does not select a single ray from an object point but rather a whole beam. We consider here the case of paraxial optics without aberrations other than defocus. In that case, during the pickup process, the rays that originate from a single object point are focused by a particular lenslet and all of them converge to a common point that is the conjugate of the object point. Unfortunately, the distance *z*′ between this focus point and the lenslet array depends on the distance between the original object point *z* and the lenslet array. This dependence is given by the lens law:

where *f* is the focal length of the lenslet array. It results from Eq. (1) that if the elemental image plane is locate at a distance *z′ _{0}* from the lenslet array, then only object points that are at the conjugate distance

*z*will appear sharp. There exists a distance interval around the plane

_{0}*z*where the objects seem to be in focus at the

_{0}*z′*-plane. That range is the so-called depth of field. Object points at

_{0}*z*out of this range will appear out-of-focus. Since the purpose of II is precisely to capture in-depth objects or scenes, this impossibility to obtain sharp images for all distances at the same time is a major drawback. It is to note that a similar problem occurs during the reconstruction stage and that the blurring of the pickup and of the reconstruction accumulate, which results in an even higher degradation of the viewing quality of the reconstructed 3-D objects. However, for the sake of simplicity, in this paper we will only consider the defocus introduced during the pickup process. We will assume that the reconstruction is either done computationally or with a pinhole array.

## 3. Using a phase mask to extend the depth of field

It has been demonstrated in previous works that there is a family of asymmetric phase masks that, when placed in the pupil of imaging systems, are useful for extending the depth of field [14]–[16]. The pupil function for an optical system that consists of lenslet and a phase mask of this kind can be mathematically represented as

where *Q(x,y)* stands for the pupil aperture (circular in our case), *x* and *y* are the spatial coordinates. The function *φ(x)* represents the one-dimensional phase profile of the phase mask and it is given by

where sgn represents the signum function, *α* is an adjustable factor that represents the maximum phase delay introduced by the mask, and *w* represents the radius of the lenslet. The exponent *k* is a design parameter that defines the phase mask order. Consequently the generalized pupil function that considers defocus is expressed as

with

as Hopkins’ defocus coefficient [21]. This coefficient can also be expressed in terms of the ideal localization of the (in-focus) object plane *z _{0}*=(1/

*z′*-1/

_{0}*f*)

^{-1}and the actual

*z*object distance [22] as

Various phase masks have been proposed to extend the depth of field (depth in the object space) or the depth of focus (depth in the image space) of classical imaging systems [14, 16], [23]–[28]. In this paper we will use the quartic phase mask (asymmetric phase mask of order *k*=4 in Eq. (3)) that has been shown to have good imaging characteristics [14].

The Optical Transfer Function (OTF) is then obtained as the autocorrelation of the generalized pupil function given in Eq. (4). Figures 1(a) and (b) present the modulus of the OTF — also known asModulation Transfer Function (MTF)—for a single lenslet and for a lenslet with a quartic phase mask respectively. These figures show curves of the MTF for various amounts of defocus given in terms of the defocus coefficient *W*
_{20}. Let us recall that for a regular lens *W*
_{20}=λ is considered as severe defocus [22]. As can be seen in this figure, the OTF for such an asymmetric phase mask has two important properties: *i*) it is mostly invariant for a large range of defocus, and *ii*) it preserves all the frequency content within its passband (it has no zero-value) even for a large amount of defocus. The first property implies that the image of an “in-focus” object is similar to the image of an “out-of-focus” object, where “in-focus” and “out-of-focus” mean close to and far from the conjugated plane of the sensor respectively. The system therefore achieves a large depth of field. However, the gain in the depth of field is at the expense of a small degradation in the visual image quality. Fortunately, the second property above means that, with a suitable digital restauration, a better image quality can be recovered. In the absence of noise, this restauration basically consists of a deconvolution operation that is done by dividing the Fourier spectrum of the image by the OTF of the phase mask [23]. This division is made possible by property *ii*) above that guarantees the absence of zeros, and by property *i*) that shows that a single deconvolution filter can be applied independently of the depth. On the other hand, it is also possible to recover an acceptable image in the presence of noise using more sophisticated restauration techniques such as Wiener filters [29, 23].

In the case of II, each lenslet suffers from limited depth of field. We therefore propose to apply a quartic phase mask to each individual lenslet. The global phase mask to be used is then defined as:

where the number of lenslets is *M*×*N* and (*x _{m},y_{n}*) are the coordinates of the center of the (

*m,n*)-lenslet.

## 4. Results

We computationally simulate an II system with 7×7 lenslets of diameter 2 mm and focal length 5 mm. The elemental image plane is placed at a distance *z′ _{0}*=100/19 mm from the lenslet array so that the object plane located at

*z*=-100 mm from the array is in focus. We use a quartic phase mask with

_{0}*α*empirically set to 35/

*π*(see Eq. (3)). Figures 2(a)–(d) show the elemental incoherent point spread function (PSF), that is the central elemental image given by a regular lenslet for a point located at

*z*=

*z*=-100 mm (

_{0}*W*

_{20}=0),

*z*=-87 mm (

*W*

_{20}=1.5λ),

*z*=-80 mm (

*W*

_{20}=2.5λ) and

*z*=-70 mm (

*W*

_{20}=4.3λ) respectively. The defocusing effect is clearly visible. Now Figs. 2(e)–(h) correspond to the same objects at the same distances but placing a quartic phase mask in front of the lenslet. In that case, it is apparent that the PSFs for the out-of-focus planes are quite similar to the PSF for the in-focus plane. However, this characteristic is obtained at the price of a small loss of resolution for the in-focus object. This effect can be seen by comparing 2(a) to 2(e). In the latter case, the central peak is wider and the sidelobes are stronger. In order to eliminate this unwanted effect, a digital restauration can be applied. For each lenslet-phase mask combination, a mean PSF is constructed by averaging the images of seven points located at regular intervals between

*z*=-70 mm and

*z*=-130 mm. The mean optical transfer function (OTF) of the combination is computed as the Fourier transform of its mean PSF. We then define the restauration inverse filter as the inverse of this mean OTF. This filter is applied to the corresponding elemental image in order to eliminate the image degradation introduced by the phase mask [23]. The results are shown in Figs. 2(i)–(l). It can be seen that the obtained PSFs of Figs. 2(i)–(k) are quite similar to each other and also similiar to Fig. 2(a) which corresponds to the in-focus PSF of a regular lenslet.

We now simulate a scene with four tumbling “E” charts at distances *z*=-280 mm, *z*=-180 mm, *z*=*z*
_{0}=-100 mm and *z*=-60 mm respectively. Figure 3(a) shows the central elemental image for a regular lenslet. Figure 3(b) shows the same for a lenslet with quartic phase mask. Figure 3(c) presents the results after restauration. This figure clearly shows the depth-of-field improvement with our proposal.

The curves in Fig. 4 show the evolution of the peak value of the PSF versus the distance of the object. The values are normalized with respect to the value of the in-focus plane. The depth of field is defined by the Rayleigh range [30] as the interval for which the normalized peak of the PSF is above √2/2. From Fig. 4, we can see that II with regular lenslets has a depth of field of about 6 mm while II with the phase mask extends it to 54 mm. Our proposal thus improves 9 times the depth of field over a traditional II system. It can be noted in Fig. 3 that quite sharp images are obtained even outside the Rayleigh range.

Lastly, Fig. 5 shows simulated 3-D reconstruction movies. The scene contains four objects located at *z*=-280 mm, *z*=-180 mm, *z*=-100 mm and *z*=-60 mm, where *z*=*z*
_{0}=-100 mm corresponds to the conjugated plane of the elemental images plane or pickup plane. Figures 5(a)–(c) correspond to a pickup with a regular lenslet array, a lenslet array with a quartic phase mask and an array with the phase mask and digital restauration respectively. In all three cases, the reconstruction step is performed using a pinhole array. It is clear from these movies that the use of a phase mask increases the depth of field of the system, even without digital restauration. However, the digital restauration step enhances the visual quality of the reconstructed object.

## 5. Conclusion

In this paper we have described the use of phase masks to substantially improve the depth of field of an integral imaging system. We particularly considered the case of the pickup stage. An array of quartic phase masks was placed in front of the lenslet array. With this modification, we showed that the point spread function of each lenslet is largely invariant to the distance of the object. Moreover, the corresponding optical transfer function has no zero within its passband, which permits a digital enhancement of the elemental images. We showed that the elemental images obtained with this technique are sharp over a large range of object distances, so that the reconstructed integral images have a better visual quality than with a regular lenslet array.

## Acknowledgments

A. Castro and Y. Frauel acknowledge financial support from Consejo Nacional de Ciencia y Tecnología (grant CB05-1-49232).

## References and links

**1. **Y. Frauel, E. Tajahuerce, O. Matoba, A. Castro, and B. Javidi, “Comparison of passive ranging integral imaging and active imaging digital holography for three-dimensional object recognition,” Appl. Opt. **43**, 452–462 (2004). [CrossRef] [PubMed]

**2. **B. Javidi and F. Okano, eds., *Three Dimensional Television, Video, and Display Technologies*, (Springer Berlin, 2002).

**3. **J.-S. Jang, F. Jin, and B. Javidi, “Three-dimensional integral imaging with large depth of focus by use of real and virtual fields,” Opt. Lett. **28**, 1421–1423 (2003). [CrossRef] [PubMed]

**4. **J.-S. Jang and B. Javidi, “Depth and lateral size control of three-dimensional images in projection integral imaging,” Opt. Express **12**, 3778–3790 (2004). [CrossRef] [PubMed]

**5. **J.-S. Jang and B. Javidi, “Large depth of focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lens and aperture sizes,” Opt. Lett. **28**, 1925–1926 (2003). [CrossRef]

**6. **M. Hain, W. von Spiegel, M. Schmiedchen, T. Tschudi, and B. Javidi, “3D integral imaging using diffractive Fresnel lens array,” Opt. Express **13**, 315–326 (2005). [CrossRef] [PubMed]

**7. **S. Jung, J. Hong, J. H. Park, Y. Kim, and B. Lee, “Depth-enhanced integral-imaging 3D display using different optical path lengths by polarization devices or mirror barrier array,” J. Soc. Inf. Disp. **12**, 461–467 (2004). [CrossRef]

**8. **M. Martínez-Corral, B. Javidi, R. Martínez-Cuenca, and G. Saavedra, “Integral imaging with improved depth of field by use of amplitude-modulated microlens arrays,” Appl. Opt. **43**, 5806–5813 (2004). [CrossRef] [PubMed]

**9. **R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, “Enhanced depth of field integral imaging with sensor resolution constraints,” Opt. Express **12**, 5237–5242 (2004). [CrossRef] [PubMed]

**10. **R. Martínez-Cuenca, G. Saavedra, M. Martínez-Corral, and B. Javidi, “Extended depth-of-field 3-D display and visualization by combination of amplitude-modulated microlenses and deconvolution tools,” J. Disp. Technol. **1**, 321–327 (2005). [CrossRef]

**11. **J.-Y Son, V. Saveljev, J.-S. Kim, S.-S. Kim, and B. Javidi, “Viewing zones in three-dimensional imaging systems based on lenticular, parallax-barrier, and microlens-array plates,” Appl. Opt. **43**, 4985–4992 (2004). [CrossRef] [PubMed]

**12. **J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics,” Opt. Lett. **27**, 324–326 (2002). [CrossRef]

**13. **A. Stern and B. Javidi, “Three-dimensional sensing, visualization, and processing using integral imaging,” Proc. IEEE **94**, 591–607 (2006). [CrossRef]

**14. **A. Castro and J. Ojeda-Castañeda, “Asymmetric phase mask for extended depth of field,” Appl. Opt. **43**, 3474–3479 (2004). [CrossRef] [PubMed]

**15. **J. Ojeda-Castañeda, A. Castro, and J. SantamaríaA. J. Glass, J. W. Goodman, M. Chang, A. H. Guenther, and T. Asakura, “Phase mask for high focal depth,” in *The 18th Congress of the
International Commission for Optics* eds., Proc. SPIE3749, 14 (1999). [CrossRef]

**16. **A. Castro and J. Ojeda-CastañedaB. W. Bowe, G. Byrne, A. J. Flanagan, T. J. Glynn, J. Magee, G. M. O’Connor, R. F. O’Dowd, G. D. O’Sullivan, and J. T. Sheridan, “Increased depth of field with phase-only filters: ambiguity function,” in
*Opto-Ireland 2005: Photonic Engineering*, eds., Proc. SPIE5827, 1–11 (2005). [CrossRef]

**17. **F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. **36**, 1598–1603 (1997). [CrossRef] [PubMed]

**18. **H. Arimoto and B. Javidi, “Integral 3D imaging with digital reconstruction,” Opt. Lett. **26**, 157–159 (2001). [CrossRef]

**19. **S.-H. Hong, J.-S. Jang, and B. Javidi, “Three-dimensional volumetric object reconstruction using computational integral imaging,” Opt. Express **12**, 483–491 (2004). [CrossRef] [PubMed]

**20. **D.-H. Shin, E.-S. Kim, and B. Lee, “Computational reconstruction of three-dimensional objects in integral imaging using lenslet array,” Jpn. J. Appl. Phys. **44**, 8016–8018 (2005). [CrossRef]

**21. **H. H. Hopkins, “The frequency response of a defocused optical system,” Proc. Roy. Soc. Lond. Series A , **231**, 91–103 (1951). [CrossRef]

**22. **J. W. Goodman, Introduction to Fourier optics, 2nd Ed. Chap. 6 (McGraw-Hill, New York, NY, 1996).

**23. **A. Castro, J. Ojeda-Castañeda, and A.W. Lohmann, “Bow-tie effect: differential operator,” Appl. Opt. **45**, 7878–7884 (2006). [CrossRef] [PubMed]

**24. **G. Häusler, “A method to increase the depth of focus by two step image processing,” Opt. Commun. **6**, 38–42 (1972). [CrossRef]

**25. **E. R. Dowski and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. **34**, 1859–1865 (1995). [CrossRef] [PubMed]

**26. **A. Sauceda, P. J. García Ramírez, L. García-González, J. Martínez-Castillo, L. Herrera-May, and A. Castro, “Imaging properties of phase shifting apodizers,” Rev. Mex. Fis. **52**, 336–341 (2006).

**27. **A. Sauceda and J. Ojeda-Castañeda, “High focal depth with fractional-power wave fronts,” Opt. Lett. **29**, 560–562 (2004). [CrossRef] [PubMed]

**28. **J. Ojeda-Castañeda, J. E. A. Landgrave, and H. M. Escamilla, “Annular phase-only mask for high focal depth,” Opt. Lett. **30**, 1647–1649 (2005). [CrossRef] [PubMed]

**29. **K. R. Castleman, *Digital image processing* (Prentice-Hall, Upper Saddle River, NJ, 1996).

**30. **A. E. Siegman, *Lasers* (University Science, Sausalito, CA, 1986).