## Abstract

Three approaches for visualization of transparent micro-objects from holographic data using phase-only SLMs are described. The objects are silicon micro-lenses captured in the near infrared by means of digital holographic microscopy and a simulated weakly refracting 3D object with size in the micrometer range. In the first method, profilometric/tomographic data are retrieved from captured holograms and converted into a 3D point cloud which allows for computer generation of multi-view phase holograms using Rayleigh-Sommerfeld formulation. In the second method, the microlens is computationally placed in front of a textured object to simulate the image of the textured data as seen through the lens. In the third method, direct optical reconstruction of the micrometer object through a digital lens by modifying the phase with the Gerchberg-Saxton algorithm is achieved.

© 2013 Optical Society of America

## 1. Introduction

Digital holography is a powerful tool for capture of transparent or semi-transparent micro-objects. The captured holograms, after being processed with the known phase retrieval or tomographic methods, yield the necessary information for surface profiling or 3D reconstruction of the refractive index distribution inside the object [1,2]. A logical continuation of the holographic capture is to display holographically the processed data in 3D with a single or multiple spatial light modulators (SLMs) taking into account the limitations and approximations arising from conversion of the 2D complex valued hologram pattern to a data format supported by the SLMs [3].

Despite recent advances in holographic displays, the proper visualization procedure for transparent objects is still an issue. Obviously, such a procedure should depend on their nature. A straightforward solution is to build a 3D point cloud of an object from the retrieved phase distribution that can then be used to create computer generated holograms for optoelectronic reconstruction of multiple perspectives. Another approach is to mimic the direct observation of a transparent object, which is positioned in front of a natural texture. To this end, the captured hologram is blended properly with the computed diffraction pattern of a given background texture, and the combined hologram is then displayed. For those objects, which give rise to strong diffraction, one could try direct reconstruction of the object beam from the captured phase-only data. In general, the recording wavelength as well as the pixel size and the number of pixels of the used CCD camera differ from the corresponding values of these parameters on the display side.

The present work reports application of three visualization approaches to holograms of pure phase micro-objects. As such silicon micro-lenses and a simulated weakly refracting 3D object with size in the micrometer range are used. The holograms of the micro-lenses are recorded in the near infrared by means of digital holographic microscopy as image-plane holograms whereas the holographic data for the simulated micro-object comprises complex amplitudes in the plane of holographic capture for multiple illumination directions. Based on the chosen visualization technique, either a single phase-only SLM or a circular configuration of multiple phase-only SLMs is used.

## 2. Description of the phase objects

We used two types of phase objects in our experiments. The first type comprises arrays of micro-lenses with a constant value of the refractive index. Figure 1 shows digital off-axis holograms for a small section from arrays of circular and square silicon micro-lenses. The circular microlens, type 35-9950-105-123, production of SUSS MicroOptics [4], is characterized with a period of 250 μm, radius of curvature 1.65 mm and numerical aperture NA = 0.18. The square microlens, type 38-5053-109-121, production SUSS MicroOptics [4], has a period of 1015 μm, radius of curvature 49 mm and NA = 0.18. The holograms were recorded by means of digital holographic microscopy at 1.28 μm as image-plane holograms at magnification 25.7 and 4.67 respectively. The detailed description of the digital holographic microscope can be found in [2,5,6]. The pixel size of the CCD camera was 30 μm, and thesize of the captured hologram was 256 pixels × 256 pixels. As is known, the silicon micro-lenses are transparent in the near-infrared region. For phase retrieval, we removed the reference wave from the hologram in the frequency domain and filtered only one of the orders. Thus we obtained the wrapped phase distribution, ${\varphi}_{W}\left(x,y\right)$, for each lens and applied a quality-guided algorithm for phase unwrapping [7]. The profilometric reconstructions of the micro-lenses are also shown in Fig. 1. Thanks to the high signal-to-noise ratio of the capture process, no smoothing of the reconstructed surface was required.

The second phase object was a simulated transparent object represented by the 3D refractive index distribution shown in Fig. 2(a), where the regions with different colors correspond to different values of the refractive index. The holographic data for this object were obtained as a result of simulation of a noiseless diffraction tomography experiment. The simulated data comprised transmission holograms recorded at plane-wave illumination of the object along different propagation directions with angular separation of 10°. The recording wavelength was 0.68 μm. Distance between the center of the object and the measurement plane was 68 μm which was equal to 100 wavelengths. The size of the optical reconstruction in the reconstruction plane at a distance of 68 μm was about 24 μm. In total, 19 complex amplitudes of the light field diffracted from the object were derived by a phase shifting approach; the four of them are shown in Fig. 2(b). The size of the holograms in pixels was 200 × 200. The size of a pixel in the measurement plane was Δ* _{hol}* = 2.4 × 10

^{−7}m. Refractive index of the medium was unity whereas the variation of refractive index within the object did not exceed 0.004 to allow the first Born approximation.

## 3. Holographic display of generated 3D point cloud objects

The 3D capture for holographic displays can be implemented by holographic means or by structured light methods with coherent or incoherent illumination [8,9]. Both approaches share common phase retrieval techniques to derive the 3D coordinates of objects points. The captured holograms or fringe patterns provide the necessary information for surface profiling or 3D reconstruction. By combining holographic and tomographic methods, one gets 3D distribution of the refractive index inside the object. The holographic methods are especially effective to capture transparent or semi-transparent objects [1]. The output from their profilometric or tomographic reconstruction can be converted into a point cloud which allows for computer generation of holograms for a given display as shown in Fig. 3. The captured data can be easily adapted to any holographic display, and this is a substantial advantage of the point cloud approach. The other benefit from this approach, when applied to tomographically reconstructed refractive index distributions, is possible visualization of interesting inner 3D regions of micro-objects. The point cloud comprises collection of self-emitting points, and the generation of holograms is usually based on a Rayleigh-Sommerfeld diffraction model to create a true 3D impact [10,11]. However, fast algorithms developed to overcome the high computational complexity of the rigorous approach can be applied to accelerate generation of holograms [12].

Following the diagram in Fig. 3, we converted the data obtained from the profilometric reconstruction of the micro-lenses and the simulated tomographic reconstruction of the transparent object in Fig. 2 into a 3D computer graphic model and then to a point cloud. We used Rayleigh-Sommerfeld diffraction model to generate multi-view phase-only holograms. For the hologram located at (*x*,*y*) plane, the complex amplitude of the light field coming from the point cloud is

*N*is the number of points in the point cloud, ${a}_{p}$ and ${\phi}_{p}$ denote the magnitude and the phase of the

*p*-th object point with coordinates $\left({x}_{p},{y}_{p},{z}_{p}\right)$, respectively, $k=2\pi /\lambda $is the wave number in radians per unit length and ${r}_{p}$is the distance between the

*p*-th object point and the point $\left(x,y\right)$ in the hologram plane:

We performed multiview optoelectronic reconstruction of the phase objects with a holographic video display system built from nine phase-only Holoeye HEO-1080P SLMs that formed a circular configuration. The detailed description of the display is given in [13]. (We used this type of a SLM, with the pixel size ${\Delta}_{SLM}=$8 μm and the number of pixels 1920 × 1080, in all experiments throughout the paper). A large beam splitter created a virtual alignment of the SLMs’ active areas without gaps between them by imaging the SLMs on one side of the beam splitter to the other side [13]. Thus a continuous increased field of view was achieved. The distance between the reconstruction volume and each SLM was 35 cm. The SLMs were slightly tilted up to position the reconstructed 3D image above the display setup and to avoid blocking of the observer’s vision by the display’s components. Good quality of the reconstructions for a tilted illumination of up to 20° has been proven by experiments and subjective test results [14]. Since the SLMs were illuminated by means of a cone mirror [13] with a single astigmatic expanding wave, $W\left(x,y\right)$, given by

*D*is the distance between the axis of the cone mirror and the SLM,

_{h}*h*

_{SLM}is the height of the SLM, and

*D*is the distance between the apex of the cone mirror and the point source of the wave positioned on the line of the cone mirror axis, the compensation for the asymmetrical illumination was taken care of. Since we used phase-only SLMs, we computed only the phase of $O\left(x,y\right){W}^{*}\left(x,y\right)$. The radius of curvature of the reference beam in the vertical plane was equal to 0.52 m whereas in the horizontal plane it was 0.27 m. To separate the image from the strong non-diffracted beam due to the pixelated nature of SLMs, we multiplied the result with $P\left(y\right)=\mathrm{exp}\left(jky\mathrm{sin}{\theta}_{t}\right)$ at θ

_{s}_{t}= 2°. The nine holograms were calculated from

*N*= 200 × 200 points for each of the phase objects to generate viewing scenes at 3 or 4° angular separation between them. Thus, a moving observer is able to see continuous 3D reconstruction within 24 or 32° viewing angle. Figure 4 depicts video recordings and enlarged parts from single frame excerpts for different viewing directions of optical reconstructions of both phase objects. For better visual perception of the micro-lenses we stretched their height profile. Note that the point cloud for the phase object in Fig. 2 consisted of the points corresponding to the outer envelope of the 3D refractive index distribution. Consequently the shown reconstruction is the visualization of the 3D shape enclosed within this envelope.

## 4. Imaging of a textured object

If we use the wrapped phase distribution, ${\phi}_{W}\left(x,y\right)$, of the lenses retrieved from the holograms in Fig. 1 for imaging of a textured background, we can detect the alterations in the original textured background image or actually “see” the lenses as we do when we look through a lens in real life. The idea is schematically depicted in Fig. 5. The complex amplitude ${L}_{W}\left(x,y\right)=\mathrm{exp}\left\{-j\phi \left(x,y\right)\right\}$ in the plane of the SLM is multiplied by the complex amplitude of the light field coming from a textured planar object positioned at some distance from the SLM plane. The phase distribution, ${\phi}_{W}\left(x,y\right)$, is retrieved from a hologram captured at λ_{1} = 1280 nm, and displayed at λ_{2} = 532 nm. We placed the textured pattern at a distance 3*f* behind the lens plane, where *f* was the focal distance of ${L}_{W}\left(x,y\right)$ at the wavelength 0.532 μm. We chose the size of the textured background to be larger than the size of the lens aperture to easily distinguish the lens circumference when we look at the textured background through the lens. The magnification in the image plane of the lens was 0.5. The block-diagram of the algorithm is presented in Fig. 6.

As the textured pattern of size 1080 pixels × 1080 pixels we used a low-pass filtered version ${T}^{\prime}\left(\tilde{x},\tilde{y}\right)$of 2D amplitude mask in the form of a regular grid of fully transparent or fully reflecting squares with the following transmission/reflection function:

_{t}= m

_{t}Δ, where m

_{t}is an integer, and Δ is the pixel size at the texture plane; the distance between the neighboring squares is also Δ

_{t}. The complex amplitude at the textured pattern plane to the right of the object is $T\left(\tilde{x},\tilde{y}\right)=t\left(\tilde{x},\tilde{y}\right)\mathrm{exp}\left\{j{\phi}_{t}\left(\tilde{x},\tilde{y}\right)\right\}$, where the phase ${\phi}_{t}\left(\tilde{x},\tilde{y}\right)$ may be added as a constant or as a uniformly distributed random value from 0 to 2π. In the first case the texture is non-diffusing, whereas the second case corresponds to a diffusing texture. To ensure the same pixel size at the texture plane, $\left(\tilde{x},\tilde{y}\right)$, the SLM plane, $\left(x,y\right)$, and the image plane, $\left(\xi ,\eta \right)$, we modeled the wavefront propagation by using Rayleigh-Sommerfeld integral and the convolution approach [8], which was implemented using DFT. We assumed that the thin lens approximation is valid, that is, we can multiply the quadratic lens phase distribution by the complex amplitude of the incident light at the lens plane. The transfer function of propagation in the free space at distance

*d*is given by

*x*and

*y*axes. As has been mentioned above, we used Holoeye SLM of size in pixels 1920×1080 and pixel period 8 μm. For this reason we increased 2 times the size of the distribution ${L}_{W}\left(x,y\right)$ by interpolation. We place the distributions ${L}_{W}\left(x,y\right)$ and $T\left(x,y\right)={\Im}^{-1}\left\{\Im \left[{T}^{\prime}\left(\tilde{x},\tilde{y}\right)\right]{G}^{-}\left(d=3f\right)\right\}$ at the SLM center which coincided with the point $\left(x=0,y=0\right)$. We use only the phase of the complex amplitude distributions ${L}_{W}\left(x,y\right)T\left(x,y\right)$ and $T\left(x,y\right)$to be fed to the corresponding areas of the phase-only SLM. We added a diverging digital lens with a focal distance ${f}_{dl}$

The reconstructed image can be obtained both numerically and optically. Before the optical reconstruction we conducted simulations of the display process with different textures. We observed the optically reconstructed focused images of the textured background at the expected reconstruction distance. Thus, we observe the lens while looking at the image of the textured background through the lens. While changing the observation position, the lens effect is clearly visible. The results of optical reconstruction with a single SLM are shown in Fig. 7 for four different textures - uniform background and *m*_{t} = 8, 16, 24.

## 5. Reconstruction of a phase object in the micrometer range

Optical reconstruction of the simulated phase object directly from the holographic data in Fig. 2(b) was a challenging task due to the very small size of the object. Note that the holograms in Fig. 1 have been captured after the light has passed through the objective of the optical microscope whereas the complex amplitudes in Fig. 2 have been derived from a simulated phase-shifting measurement without any magnification.

The reconstruction distance for the holograms in Fig. 2 for the pixel period Δ_{SLM} = 8 μm and wavelength 0.532 μm becomes 0.096 m, and the lateral size of the reconstructed object is 800 μm for a viewing direction of 70°. (We chose for illustrations the viewing direction of 70° as it corresponded to the maximum size of the object along the horizontal axis). For the numerical reconstruction using the full complex amplitude, $O\left(x,y\right)={a}_{O}\left(x,y\right)\mathrm{exp}\left\{jk{\phi}_{O}\left(x,y\right)\right\}$, of the object beam in the hologram plane, we obtained high-contrast intensity images as the one shown in Fig. 8(a) of different views of a concise 3D shape which rather closely resembled the 3D refractive index distribution in Fig. 2. One should take into consideration that the transmission holograms in Fig. 2 and hence the reconstruction in Fig. 8 (a) are influenced also by the inner parts of the 3D refractive index distribution. However, when we omitted the amplitude information and used only the phase, we obtained completely unsatisfactory result shown in Fig. 8(b). The phase information was not enough to yield the correct intensity distribution at the image plane, and the reconstructed image was severely distorted.

To solve the problem we applied the Gerchberg-Saxton algorithm [15,16] to modify iteratively the phase at the hologram plane (*x*,*y*) knowing the correct complex amplitude

*K*= 30 iterations is shown in Fig. 10(a) for the viewing direction of 70°. As it can be seen, the quality of the reconstruction from the modified phase is satisfactory. The 3D shape is clearly seen as in Fig. 8(a); the only drawback, when compared to the reconstruction in Fig. 8(a), is the slight increase of the noise at the area surrounding the reconstructed image.

To increase the 3D shape size in the optical reconstruction we incorporated a digital magnifying lens at the SLM plane, and calculated the required phase hologram by a Fresnel approximation. The optical reconstruction is shown in Fig. 10(b) for the reconstruction distance 1.75 m. The size of the reconstructed object is about 6 mm which gives about 250 times magnification in comparison with its original size.

## 6. Conclusion

3D holographic display of transparent objects is achieved. The object may have a constant or varying refractive index. Different visualization modes are designed, implemented and tested. In one mode, the 3D point cloud structure is constructed from captured holographic data, and then the computer generated holograms of this 3D structure are formed and holographically displayed using a circular holographic display consisting of multiple phase-only SLMs. This display provides seamless viewing of the 3D point-cloud structure from continuous directions within a rather large viewing angle [13]. In another mode, a real-life viewing through transparent objects is imitated. An artificial planar textured pattern is generated and inserted as the background of the uniform refractive index transparent object to be visualized. Diffraction signals corresponding to the combined 3D scene (planar textured background and the 3D transparent object in front of it) are computed and fed into a phase-only SLM for holographic reconstruction. In the third approach, the captured holographic data related to a 3D transparent object a directly used to retrieve the complex amplitude of a diffracted light distribution as it passes and exits the transparent object. It is observed that only the phase component of this complex amplitude, obtained by simply discarding the magnitude, does not provide sufficient quality 3D reconstructions using a phase-only SLM as the holographic display device. Therefore, a more sophisticated procedure based on the iterative Gerchberg-Saxton algorithm is employed to compute the phase distribution to be written on the phase-only SLM. As a consequence, a satisfactory naked-eye 3D holographic display of a transparent (varying index) micro object is achieved, with a magnification of about 250.

## Acknowledgments

This work is supported by the European Community (EC) within the Seventh Framework Programme (FP7) under Grant 216105 with the acronym Real 3D. This work was partly supported by the IT R&D program of MSIP/KEIT [K10045289, Development of Virtual camera system with real-like contents].

## References and links

**1. **E. Cuche, F. Bevilacqua, and C. Depeursinge, “Digital holography for quantitative phase-contrast imaging,” Opt. Lett. **24**(5), 291–293 (1999). [CrossRef] [PubMed]

**2. **Y. Sung, W. Choi, C. Fang-Yen, K. Badizadegan, R. R. Dasari, and M. S. Feld, “Optical diffraction tomography for high resolution live cell imaging,” Opt. Express **17**(1), 266–277 (2009). [CrossRef] [PubMed]

**3. **L. Onural, F. Yaras, and H. Kang, “Digital holographic three-dimensional video displays,” Proc. IEEE **99**(4), 576–589 (2011). [CrossRef]

**4. **SUSS MicroOptics, http://www.suss-microoptics.com/

**5. **F. Charrière, A. Marian, F. Montfort, J. Kuehn, T. Colomb, E. Cuche, P. Marquet, and C. Depeursinge, “Cell refractive index tomography by digital holographic microscopy,” Opt. Lett. **31**(2), 178–180 (2006). [CrossRef] [PubMed]

**6. **F. Charrière, J. Kühn, T. Colomb, F. Montfort, E. Cuche, Y. Emery, K. Weible, P. Marquet, and C. Depeursinge, “Characterization of microlenses by digital holographic microscopy,” Appl. Opt. **45**(5), 829–835 (2006). [CrossRef] [PubMed]

**7. **D. Ghiglia and M. Pritt, *Two-Dimensional Phase Unwrapping* (*J. Wiley & Sons*, 1998).

**8. **U. Schnars and W. Juptner, *Digital Holography* (Springer, 2005).

**9. **E. Stoykova, A. Alatan, P. Benzie, N. Grammalidis, S. Malassiotis, J. Ostermann, S. Piekh, V. Sainov, C. Theobalt, T. Thevar, and X. Zabulis, “3D Time-Varying Scene Capture Technologies – A Survey,” IEEE TCSVT **17**(11), 1568–1586 (2007).

**10. **J. P. Waters, “Holographic image synthesis utilizing theoretical method,” Appl. Phys. Lett. **9**(11), 405–407 (1966). [CrossRef]

**11. **J. W. Goodman, *Introduction to Fourier Optics* (McGraw-Hill, 1996).

**12. **H. Kang, T. Yamaguchi, H. Yoshikawa, S. C. Kim, and E. S. Kim, “Acceleration method of computing a compensated phase-added stereogram on a graphic processing unit,” Appl. Opt. **47**(31), 5784–5789 (2008). [CrossRef] [PubMed]

**13. **F. Yaraş, H. Kang, and L. Onural, “Circular holographic video display system,” Opt. Express **19**(10), 9147–9156 (2011). [CrossRef] [PubMed]

**14. **F. Yaraş, “Three-dimensional holographic video display systems using multiple spatial light modulators,” Ph.D.dissertation (Bilkent University, 2011).

**15. **R. Gerchberg and W. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik (Stuttg.) **35**, 237–246 (1972).

**16. **G. Liu and P. Scott, “Phase retrieval and twin-image elimination for in-line Fresnel holograms,” J. Opt. Soc. Am. A **4**(1), 159–165 (1987). [CrossRef]