A new kind of plenoptic imaging system based on Laser Optical Feedback Imaging (LOFI) is presented and is compared to another previously existing device based on microlens array. Improved photometric performances, resolution and depth of field are obtained at the price of a slow point by point scanning. Main properties of plenoptic microscopes such as numerical refocusing on any curved surface or aberrations compensation are both theoretically and experimentally demonstrated with a LOFI-based device.
© 2013 OSA
In conventional imaging setups, when impacting the detector, only the position of incoming photons is recorded while the information on the angle of arrival is lost. This results in a severe loss of information (stereoscopic information for instance). To handle that problem a first theoretical solution was introduced by the French Nobel Prize Gabriel Lippmann in 1908 under the name “photographie totale” , later called “light field” or “plenoptic imaging”. Contrary to classical imaging setups, plenoptic imaging is a technology where both spatial and angular information are recorded. This property leads to unusual features in imaging such as the possibility to numerically refocus a blurry photo after the picture has been taken. This results in an extended depth of field without reducing the aperture of the objective lens and thus there is no need for increasing the exposure time i.e. no movement blurring . The opportunity to correct the objective’s aberrations by digital post-processing has also been demonstrated . One of the first experimental device based on a microlens array was presented by Levoy et al. [2,3] but suffers from several drawbacks like a low sensitivity and a trade-off between the resolution and the latitude of refocusing.
In the first part of this paper a new kind of plenoptic microscope based on Laser Optical Feedback Imaging (LOFI) is presented  and compared to the previous conventional technology (using microlens array). In this setup, the latitude of refocusing no longer depends on the resolution. Next to that, this imaging setup also has the advantage of being shot noise limited (ultimate sensitivity) [5,6] and is thus suitable for optically resolved imaging through scattering media where a few ballistic photons are available. These benefits are at the price of a slow acquisition caused by point by point scanning. In the two last parts, numerical aberration compensation and refocusing on a curved surface are theoretically and experimentally demonstrated with LOFI.
2. Theoretical background: defocusing
LOFI microscope  (see Fig. 1 ) is an ultra-sensitive laser autodyne interferometer imaging technique combining the high accuracy of optical interferometry with the extreme sensitivity of class B lasers to optical feedback . Photons are emitted, frequency shifted by a value Fe, retro-reflected by the target and finally reinjected into the laser cavity. In this autodyne method, an optical beating between the reference wave and the signal wave (the light back-reflected by the target) takes place inside the laser cavity. This beating at the frequency shift Fe of the laser is detected by a photodiode and demodulated, leading to amplitude and phase information on the reinjected electric field. Images are obtained because of a point by point scanning by galvanometric mirrors . Because one sets the total round-trip frequency shift Fe close to the relaxation frequency of the laser, a large amplification of the optical beating inside the cavity by the laser gain is achieved. More precisely, it was previously shown that the detection is shot noise limited even at low power (~mW) [5,6] which is well suited for the exploration of biological media (no damage and high optical attenuation of the tissues). In addition, because the laser plays the role of both the emitter and the detector it results as a self-aligned and easily transportable device.Eq. (1) and 2, one can see that information on both the position and the direction of propagation of retroreflected photons is accessible (the wavefront contains a complete information).
On the contrary to classical imaging device (Fig. 2(a) ) where only the position of impacting photons is recorded (one pixel = one position) by a Charge Coupled Device (CCD), imaging setup showing such properties are called total, light field or plenoptic. A first configuration was presented by Ren Ng et al. in Stanford University . In the case of Ng’s setup (Fig. 2(b)) a microlens array is placed just before the CCD, each microlens covering N pixels. The impacting photons position are thus sampled by each microlens while the angle of arrival of the photon is sampled by the N pixels behind each microlens (see Fig. 2(b)).
This double spatial/angular information allows obtaining images with unusual properties such as the possibility to refocus a blurry image after raw acquisition. Indeed, in a defocused image, photons incoming from a punctual target impact different pixels on the detector instead of arriving at the same pixel. But because they arrive with different angles too (Fig. 2), the image can be numerically refocused by summing these light rays incoming from the target.
In the case of the conventional plenoptic setup based on microlens array (Fig. 2), as each pixel corresponds to one light ray – i.e. position plus angle of a photon -, the refocusing is possible by simply summing the N appropriates pixels . Ng showed that with this refocusing technique the equivalent depth of field is multiplied by a factor N: this is related to the angular resolution, the aperture of the objective is sampled by N elements. This device however suffers from a trade-off between the spatial and the angular resolution which depends on the choice of N. An increase of N increases the “plenoptic property” that is to say the angular resolution but reduces the final resolution (the size of the microlens is proportional to N).
In the case of LOFI setup (Fig. 1), the defocus is visible in Eq. (2) in the second term and corresponds to the quadratic phase dependence of the plane waves in the signal. To numerically refocus raw images, this phase has to be cancelled by multiplying the signal in Fourier space (Eq. (2)) by the phase filter Hfilt(L,ν,μ):4]). After filtering and inverse Fourier transform in spatial domain, one has the following final synthetically refocused signal:4] (the coupling of the reinjected electric field with the laser cavity acts as a confocal pinhole). This last problem limits the accessible latitude of refocusing but is partially compensated by the ultimate shot noise sensitivity of LOFI [5,6] (this is an important advantage over the low sensitive CCD detector of conventional plenoptic setups).
3. Compensation of aberrations
Aberration compensation is an important concern for imaging through heterogeneous biological media  or for obtaining highly resolved images with a cheap objective. To correct aberrations, the first solution consists in the introduction of adaptative optics resulting in an aberration-free laser spot in the target plane. Spatial light modulators or deformable mirrors [8,9] can be used in this way. Another way to handle that problem is to use a plenoptic detector, the compensation is made after the recording of the raw image by a numerical post-processing. This capability has already been demonstrated with conventional plenoptic setups with benefits both in the field of photography  as well as in microscopy . More precisely, defocus is just one particular second order (relatively to the aperture) aberration; if one totally controls light field rays, there is no reason that higher order aberrations could not been corrected too. In what follows we demonstrate aberration compensation in a LOFI plenoptic imaging setup. To simplify the situation, constant aberration in the image field - i.e. the raw PSF does not depends on the field - is first considered, Eq. (2) is turned into:Eqs. (5) and (6) is now given by the following simplified expression:
To illustrate the capabilities of LOFI technique and to compare it to the theory, spherical silica beads of 40 μm diameter (Fig. 3(a) ) are imaged by the LOFI setup depicted on Fig. 1. The advantage of this object is that a single bead acts as a punctual reflector (Dirac) located at its center. The raw PSF (Eq. (1)) is then directly accessible. Contrary to the paraxial lens used in , an objective (lens L4 of Fig. 1) introducing large aberrations is now placed in the setup. This objective is a simple plano-convex spherical lens of focal length equal to 8 mm and is voluntarily tilted to induce important asymmetrical aberrations relatively to the optical axis (astigmatism and coma). Figure 3(b) shows a raw image with a defocus L ~300 μm resulting in a spot size of 100 μm in the target plane. The numerical aperture of the beam is equal to 0.25 which implies an expected ultimate resolution of r / √2 ≈1 μm in the case of totally corrected aberrations (after numerical refocusing). From raw image of Fig. 3(b), the defocus is first removed which leads to Fig. 3(c) where only remain astigmatism, coma and higher order aberrations. Then astigmatism and coma are removed leading to Fig. 3(d). This final image can be compared to the image (Fig. 3(e)) obtained with an aberration-free objective of close numerical aperture and resolution (NA ≈0.15 and r / √2 ≈1.5 μm). To properly correct the defocus and compensate the aberrations it is necessary to evaluate precisely the defocus and the amount of astigmatism and coma (i.e. evaluating L, A and B, see Eq. (7)). Best coefficients are evaluated by minimizing the sum of the amplitude of each pixel. The pertinence of this criteria was firstly demonstrated by Dubois et al in  for autofocus and here the possibility to extent it to aberration compensation is experimentally proved. This criteria is in fact related to the minimization of phase scrolling in the final PSF which explains that it works with all aberrations in general, including defocus. To finish, on Fig. 3(f) |hSA(L,x,y)*haber(L,x,y)| is represented with haber the inverse Fourier transform of Haber (Eq. (6), A, B and L are obtained with ). This illustrates the compensation of astigmatism and coma which were initially present in the objective.
By comparing Figs. 3(c) and 3(d), an important improvement in the image quality can be observed, which confirms the interest of our numerical aberration compensation technique. However by comparing Figs. 3(d) and 3(e), it is also possible to see that these corrections are not totally perfect. Differences between these images can be essentially explained by higher order non-compensated aberrations.
4. Refocusing on a surface
Getting a sharp image of a 3D object everywhere in the field (i.e. increasing the depth of field while keeping both high resolution and large aperture) is not a new issue. The simplest solution is called focus stacking ; in this technique, several images are acquired, each corresponding to a different plane of focus. This stack of images is then computed in order to select in each zone of the field the sharpest image; this results in a globally sharp image everywhere in the field. The problem with that technique is the important amount of time and data resources required. Another solution is the use of non diffractive beam which automatically results in an extended depth of field. Such beams can be obtained with axicons , the accessible depth of focus is however limited by the lens size.
Here we propose to use plenoptic imaging, with the advantage that only one raw acquisition is needed. The refocus distance now simply depends on the image field. However, as it was stated before, the depth of field extension is limited up to N (the number of pixels behind each microlens) with a conventional plenoptic setup (Fig. 2(b)). The use of the new LOFI plenoptic setup is thus interesting. In the previous case of a plane target perpendicular to the optical axis situated at a distance L of the laser waist (see Fig. 1), the numerical refocusing is simply obtained by a convolution product of the raw signal sR(x,y) with the refocusing filter hfilt(L,x,y) equal to the inverse Fourier transform of Hfilter(L,ν,μ) (see Eqs. (2)-Eq. (4)):Eqs. (2) and (4)) and to use a fast multiplication (Eq. (3)) rather than making convolution product directly (Eq. (8)). Indeed in the first situation the computation time is only proportional to n log(n) (n the total number of pixel in the processed image) whereas it is proportional to n2 for the second one.
When the ratio laser waist / target distance depends on the field, things are different and the convolution of Eq. (8) of raw defocused image sR(x,y) is now performed with a refocusing filter which changes with the field (L now depends on x and y). More quantitatively, Eq. (8) becomes:Eq. (9) is the excess computation time similar to the convolution product proportional to n2. For example for a 512*512 pixels image it is close to one hour with a 3 GHz dual core processor. In order to reduce computation time, another processing is used: from raw signal, images refocused in several planes spaced by the Rayleigh distance are calculated using quick calculation described in Eqs. (2), (3) and (4). One just has to select in each region of the field the sharpest image exactly like in focus stacking, except that images of the stack are calculated from one raw image. Finally, the computation time is proportional to n log(n) and reduces from one hour down to 30 seconds.
In order to get an experimental demonstration, the object of Fig. 3(a) is used again but now the film where beads are glued is curved in the y direction and simply tilted in the x direction. The objective lens is the same than in Fig. 3(e) (no aberrations). Raw image is presented on Fig. 4(a) with a defocus comprised between 250 and 1000 μm depending on the field. After numerical refocusing on an intermediate plane corresponding to L = 750 μm (see Fig. 4(b)), one can observe that only a small number of beads are sharp. These sharp beads are distributed along an arc of a circle because of the curved-tilted support film. To conclude Fig. 4(c) shows the best focusing surface and Fig. 4(d) represents the final computed image. In this last image, all beads are sharp despite their initial different positions along optical axis. Because of a Rayleigh distance ZR ~10 μm, the calculation of 50 stack images in intermediate plane is needed.
Once again, the optimal surface of refocusing shown in Fig. 4(c) is calculated locally with the help of the Dubois’s criteria  (minimization of the sum of pixels amplitude). More precisely, the best refocusing distance is evaluated on four zones in the images and from these measurements, an optimal surface is fitted by a second order polynomial function L(x,y) of equation ax + by2 + cy + d (in x direction, the film is simply tilted) leading to Fig. 4(c). A restriction to the second order is in agreement with the smooth curvature of the target. Of course if the surface were unknown, the best fitting polynomial function could change but the approach would have been the same. However it is noteworthy that low quality bead images are obtained on the edge of Fig. 4(d). This is due to both an incomplete scan of these beads (not all plane waves are present) and to a non perfectly planar laser scan which acts as a phase drift. These two effects results in a degradation  of the final image in the radial direction.
Plenoptic imaging setups are devices having the ability to record both position and angle of arrival of photons incoming on a detector. In this paper, a new kind of plenoptic microscope based on LOFI technique is demonstrated and its performances are compared with those of a previously existing device based on microlens array. It is shown that one benefits from a shot noise ultimate sensitivity limitation at low laser output power of 10 mW. This new plenoptic microscope is thus well suited for imaging through fragile scattering media (like biological media). Another advantage of LOFI over microlens array techniques is an extended depth of focus and an absence of trade-off spatial / angular resolution. However this is possible at the price of slow point by point galvanometric scanning and of degradation of photometric performances when the target is not in the objective’s imaging focal plane. Unique possibilities related to plenoptic properties are also demonstrated in the paper like numerical compensation of constant aberration (astigmatism and coma of the objective) or refocusing on a curved surface after the acquisition. In the future, we plan to extend this work by implementing a correction of local aberrations beyond local defocus. Such an improvement will enable aberrations-free images through both scattering and heterogeneous media, which paves the way to biological sample imaging .
References and links
1. G. Lippmann, “Epreuves réversibles. Photographies intégrales,” Comptes Rendus De l'Académie Des Sciences De Paris 146, 446–451 (1908).
2. R. Ng, Digital light field photography,” Ph.D. Thesis, University of Standford (2006), http://www.lytro.com/renng-thesis.pdf.
3. M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235(2), 144–162 (2009), http://graphics.stanford.edu/papers/lfillumination/levoy-lfillumination-jmicr09-lores.pdf. [CrossRef] [PubMed]
4. W. Glastre, O. Jacquin, O. Hugon, H. Guillet de Chatellus, and E. Lacot, “Synthetic aperture laser optical feedback imaging using a translational scanning with galvanometric mirrors,” J. Opt. Soc. Am. A 29(8), 1639–1647 (2012). [CrossRef] [PubMed]
5. E. Lacot, O. Jacquin, G. Roussely, O. Hugon, and H. Guillet de Chatellus, “Comparative study of autodyne and heterodyne laser interferometry for imaging,” J. Opt. Soc. Am. A 27(11), 2450–2458 (2010). [CrossRef] [PubMed]
6. W. Glastre, O. Jacquin, O. Hugon, H. Guillet de Chatellus, and E. Lacot, “Deep and optically resolved imaging through scattering media by space-reversed propagation,” Opt. Lett. 37(23), 4823–4825 (2012). [CrossRef] [PubMed]
7. R. J. Zawadzki, B. Cense, Y. Zhang, S. S. Choi, D. T. Miller, and J. S. Werner, “Ultrahigh-resolution optical coherence tomography with monochromatic and chromatic aberration correction,” Opt. Express 16(11), 8126–8143 (2008). [CrossRef] [PubMed]
8. A. J. Wright, S. P. Poland, J. M. Girkin, C. W. Freudiger, C. L. Evans, and X. S. Xie, “Adaptive optics for enhanced signal in CARS microscopy,” Opt. Express 15(26), 18209–18219 (2007). [CrossRef] [PubMed]
10. F. Dubois, C. Schockaert, N. Callens, and C. Yourassowsky, “Focus plane detection criteria in digital holography microscopy by amplitude analysis,” Opt. Express 14(13), 5895–5908 (2006). [CrossRef] [PubMed]
11. CombineZP (2010), available at http://hadleyweb.pwp.blueyonder.co.uk/CZP/News.htm.
12. Z. Zhai, S. Ding, Q. Lv, X. Wang, and Y. Zhong, “Extended depth of field through an axicon,” J. Mod. Opt. 56(11), 1304–1308 (2009), http://www.tandfonline.com/doi/abs/10.1080/09500340903082689. [CrossRef]
13. W. Glastre, O. Jacquin, O. Hugon, H. Guillet de Chatellus, and E. Lacot, “Limitations of synthetic aperture laser optical feedback imaging,” J. Opt. Soc. Am. A 29(11), 2247–2255 (2012). [CrossRef] [PubMed]