Open Access
Add to BibSonomy
Abstract
Adaptive optics can improve the performance of optical systems and devices by correcting phase aberrations. While in most applications wavefront sensing is employed to drive the adaptive optics correction, some microscopy methods may require sensorless optimization of the wavefront. In these cases, the correction is performed by describing the aberration as a linear combination of a base of influence functions, optimizing an image quality metric as a function of the coefficients. The influence functions base is generally chosen to either efficiently represent the adaptive device used or to describe generic wavefronts in an orthogonal fashion. A rarely discussed problem is that most correction bases have elements which introduce, together with a correction of the aberration, a shift of the imaging field of view in three dimensions. While simple methods to solve the problem are available for linear microscopy methods, nonlinear microscopy techniques such as multiphoton or second harmonic generation microscopy require non-trivial base determination. In this paper, we discuss the problem, and we present a method for calibrating a shift-less base on a spatial light modulator for two-photon microscopy.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Adaptive optics has proven to be an effective method in microscopy [1] for correcting both sample-induced and system-induced optical aberrations during imaging, improving resolution, contrast and imaging depth of most microscopy methods, which is particularly crucial in deep brain imaging [2]. In an ideal situation wavefront-sensor based systems [3] can rapidly provide a measurement of the aberration to be corrected. However, several factors limit the possibility of using wavefront sensors in microscopy, such as the technical difficulty of implementation, the unavailability of bright point-like sources within three-dimensional biological samples, as well as the limited field of view of validity of the correction. Due to these limitations, sensorless adaptive optics [4] is often preferred in applications where the ease of implementation and wider fields of view take priority over the correction speed.
Sensorless adaptive optics is generally performed by optimizing a numerical image metric which is maximum (or minimum) when the aberration is fully corrected. The aberration $\Phi$ is defined as a linear combination of $N$ elements of an orthogonal base
so that the aberration $\Phi$ is entirely defined by the vector of coefficients $ {\textbf {a}}=\{a_1,\ldots , a_n \}$. The base $B=\{ \phi _1,\ldots , \phi _N\}$ can vary depending on the application: common examples are the Zernike polynomials base or the influence functions of the actuators of a deformable mirrors. A variety of optimization algorithms can be employed in order to find the set of coefficients for $ {\textbf {a}}$ optimizing the metric, such as hill-climbing methods [5], model-based methods [6], nonlinear optimizations [7] or neural networks [8]. An important aspect for successful optimization lies in the selection of the aberration base $B$ chosen for optimization. Previous studies have shown how, while any generic base can be used for optimization, gradient orthogonal bases [6,9] (e.g. Lukosz Polynomials) have several beneficial effects for optimization, allowing for particularly fast model-based algorithms to be implemented, and importantly guaranteeing that the geometrical center of the point spread function (PSF) of the system remains invariant during optimization, and therefore the acquired images are not spatially shifted. While several definitions can be employed, in order to define the geometrical center of the PSF we will, throughout this manuscript, use the PSF intensity distribution $PSF(x,y,z)$, and the definition of center of mass $(COM)$ at a given axial depth $z$: and define the axial position of the center $C_z$ of the PSF as the coordinate for which the second moment $SM(z)$ of $PSF(x,y,z)$ is minimum: withWe then define the lateral coordinates of the PSF center as the center of mass in the $C_z$ axial plane:
The shift-less nature of corrections through a gradient orthogonal base can be proven mathematically for linear imaging methods [10], where the PSF is effectively represented by the distribution of light intensity at the focal plane of the system. As shown in Fig. 1, this invariance does not hold true for two-photon microscopy, or in general for methods with a PSF non-linearly proportional to light intensity (e.g. second/third harmonic generation imaging, Raman imaging [11], three-photon fluorescence microscopy [12]).
Fig. 1. Simulation of point spread function in both lateral and axial planes, for Zernike and Lukosz polynomials describing coma-like and spherical-like aberrations. Aberrations are simulated for A) linear imaging methods and B) quadratic imaging methods (e.g. two-photon microscopy, second harmonic generation imaging). It should be noticed how, for linear PSFs, Lukosz polynomial aberrations produce no shift in the centre of the PSF as defined in Eqs. (2), (3) and (4), due to the gradient orthogonal nature of the base. This does not hold true for nonlinear imaging, as reported in panel B.
While small shifts in the field of view may be considered negligible, a conventional aberration base can lead to significant problems during optimization. A typical problematic situation arises when optimizing signal on a focal plane with dim features, directly above or below planes containing brighter objects. In these cases, optimizations may introduce significant amounts of aberrations, shifting the image field of view towards the brighter features, instead of correcting the aberration on the plane of interest. Moreover, recent research [13–17] has shown how imaging on wide fields of view can significantly benefit from applying space-varying corrections on small subregions of the field of view. In these applications, a shift-less correction base is extremely important in order to avoid significant artifacts when merging the subregion images in a single output image.
This manuscript presents a calibration method for the determination of a shift-less correction base for adaptive optics in nonlinear microscopy setups. The calibration was performed on a custom setup for multiphoton anisoplanatic adaptive optics imaging and optogenetics, but can easily be adapted to any microscope equipped with an adaptive optical element for aberration correction (e.g. deformable mirror, spatial light modulator). The correction base was tested on a standard hill-climb optimization algorithm, both on a synthetic sample of microspheres and in in-vivo imaging experiments.
2. Methods
2.1 Experimental setup
The only modification required to perform the calibration of the base on a traditional purpose-built nonlinear microscopy setup is the addition of a pixelated detector (e.g. CMOS or CCD camera) conjugated to the image plane of the system in a fluorescence emission light optical path. This can generally easily be achieved by temporarily substituting one of the regular detectors of the microscope with a tube lens followed by the camera. In the specific case of these experiments, however, a slightly more complex setup was employed, intended for holographic random access imaging [18], anisoplanatic adaptive optics imaging [13] and photostimulation. This more complex setup is however not required to replicate the base calibration procedure reported in this manuscript, for which a traditional detector and an inexpensive camera would suffice.
The setup, as schematically represented in Fig. 2, panel B, uses as a light source a titanium-sapphire laser (Chameleon Ultra II, Coherent), expanded to a $1/e^2$ radius of $6 \: mm$. The beam impinges on a liquid cristal on silicon spatial light modulator (31 Hz refresh rate, 1,152 by 1,920 pixels, 9.2 $\mu m$ pixel pitch, Meadowlark, USA). The SLM is conjugated with the plane between a couple of galvanometric mirrors (GVS-012/M, Thorlabs, USA) by a 4-f telescope (AC-508-200-B and AC-508-150-B, Thorlabs), with a physical point blocker in the focal plane of the first lens, blocking the zeroth diffraction order of the SLM. A second 4-f telescope (AC-508-180-AB and AC-508-400-AB, Thorlabs, USA) conjugates the galvanometric mirrors with the back aperture of the objective (XLUMPlanFL N, 20X, 1.0 NA, Olympus, Japan). A long-pass dichroic mirror (FF665-Di02-25x36, Semrock, USA) is positioned between the SLM and the galvanometric mirrors, separating fluorescence light from the excitation laser in a descanned optical path. After further filtering with a short-pass filter (FF01-680/SP-25, Semrock, USA) and an appropriate bandpass filter, fluorescence light is collected on an EMCCD camera (Hnu 128 AO, Nuvu, Canada) through an adjustable magnification lens ($12-72\: mm$, 1.2 f #, Cosina, Sony, Japan).
Fig. 2. Methods figures: A) Simulation of displacement of the PSF center as defined in Eqs. (4), (6). Blue lines represent axial displacement, green and orange lines represent lateral displacement. B) Schematic representation of the experimental setup, not to scale. SLM, spatial light modulator; DM, dichroic mirror; GM, galvanometric mirror; OBJ, objective; CCD, detector. C) Dataset for the calibration of $S_{3,-1}$. D) Dataset for the calibration of $S_{4,0}$.
The SLM was employed as the adaptive optical element of the system, projecting phase patterns equal to the desired aberration phase, summed to a tilt pattern displacing the laser focus laterally, in order to avoid it being blocked by the zeroth-order remover. The camera was employed both as a image plane detector for base calibration, and, by integrating on a small subregion around the focus image, as a "single pixel" detector for two-photon imaging.
2.2 Base determination procedure
In order to calibrate a shift-less correction base, the Zernike polynomials base $Z_{n,m}$ [19] was chosen as a starting point. This selection is due to the fact that, as empirically shown in the simulations in Fig. 2, panel A, the center of the PSF is shifted laterally only for coma-like polynomials (with odd $n$ coefficient and $m=\pm 1$), and axially for radial (even $n$ and $m=0$) polynomials. This allows to define a shift-less base $S$ of polynomials $s_{n,m}$ as:
For each value of $a$, the curve was smoothed with a Savitzky-Golay filter, and interpolated with a polynomial to find its minimum value. The value of the $s_{n,1}$ coefficient is calculated as the slope of the fitted line passing through all minima. The identical procedure was applied for vertical coma-like polynomials to find the coefficients $s_{n,-1}$ with odd $n$.
A similar procedure was applied to determine the coefficients $s_{n,0}$ with even $n$, representing radial aberrations. In this case, aberrations in the form:
were introduced, and instead of measuring the displacement of the center of mass, the second moment of the intensity distribution was computed according to Eq. (5). The obtained dataset is represented in Fig. 2, panel D, and the same interpolation and fitting procedure, as for the previous two coefficients, was employed to determine the coefficients $s_{n,0}$ with even $n$.2.3 Sensorless correction method
In order to test the shift-less nature of the calibrated base, as well as comparing its performance with a more conventional base, three-dimensional images of two-photon fluorescence were acquired on a sample of fluorescent microspheres embedded in agarose gel, and by performing imaging in-vivo in the barrel cortex of anesthetized mice. The base was calibrated on our setup up to the sixth aberration order. Optimizations were performed on all elements of the base, excluding Piston, Tip, Tilt and Defocus. Sensorless optimization was performed with an hill-climbing algorithm, as fast convergence times and/or optimal correction performance were not critical for the experiments. During correction, the excitation foci were scanned on the sample in a spiral pattern, with a period equal to the camera exposure. The average intensity of the fluorescent spots was employed as a metric for optimization. For each of the polynomials of the selected optimization base, the aberration amplitude was linearly modulated, measuring the intensity metric at each step of the modulation. The metric function was filtered with a Savitzky-Golay filter and interpolated. The peak-to-valley polynomial amplitude maximizing the metric value was chosen as the correction coefficient.
2.4 In-vivo imaging preparation
All animal procedures were performed according to the guidelines established by the Italian Council on Animal Care, and were approved by Italian Government Decree No. 947/2017-PR. Every effort was made to minimise the number of mice used, and their suffering. A single female six months old C57BL/6 mouse was used for this study. The mouse was anesthetized with 2,2,2-tribromoethanol (Sigma) and mounted in a stereotaxic frame. Craniotomy and durotomy were performed with $3\: mm$ diameter biopsy punch in the parietal lobe of the left hemisphere. $2 \: \mu L$ of a 10 $\mu M$ solution of sRhoVR1-GP1 [20] (a synthetic fluorophores with high voltage sensitivity, excellent two-photon performance, and compatibility in intact mouse brains) in ACSF was injected with a 26-gauge blunt tip Hamilton syringe ($1 \: \mu L/min$). Here, sRhoVR1 was prepared in-house following a modified procedure that will be reported elsewhere. The injection site was at the centre of craniotomy, $1\: mm$ under the surface of the brain. The needle was left in place for 10 minutes before withdrawal. A cranial window composed of two coverslip glasses ($3\: mm$ and $4\: mm$ diameters, respectively) was placed over the brain and attached with cyanoacrylate glue. Then, when the glue was completely dry, a metal head plate was placed over the skull with bicomponent epoxy glue. Imaging was performed 8 days after surgery. The mouse was positioned under the microscope and kept anesthetized with 2,2,2-tribromoethanol for 1 hour.
3. Results
For both fluorescent microbeads and in-vivo imaging, three separate three-dimensional datasets were acquired: one without any applied correction, one using the Zernike polynomials as a correction base, and one using the shift-less base calibrated as reported in the methods section. To ensure the fairest possible comparison and prevent motion artefacts from invalidating results, all datasets were acquired first without correction, then correcting with the Zernike base, followed by the shift-less base, and finally a second uncorrected dataset was acquired to ensure no significant sample drift appeared during the experiment.
Figure 3 reports results for microbeads experiments. Experiments were performend on $\approx 200\:nm$ diameter green-fluorescent microbeads, embedded in a $10 \% \: w/v$ agarose gel. Images were acquired approximately $500\: \mu m$ inside the agarose gel sample, where the sample-induced aberration was dominated by a spherical-like component due to index mismatching. Three-dimensional datasets with $200$ by $200$ by $500 \: nm^3$ size voxels were acquired over the same $20$ by $20$ by $10 \: \mu m^3$ field of view for all three correction cases. Panel A of the Fig. 3 shows the two-dimensional plane at the axial center of the three-dimensional datasets. It can immediately be noticed how, while correction with a shift-less base only increases the average intensity of the image, the details in the field of view are significantly changed when optimizing with a Zernike base, due to an axial shift in the imaging plane. Panel B reports, for each dataset, the two-dimensional plane with highest correlation with the uncorrected original image, which for the shift-less base case is found at only $1\: \mu m$ from the dataset’s axial center, while for the Zernike base is at the edge of the dataset, at $4\: \mu m$ from the dataset’s axial center. Most importantly, even when recovering the original plane in the Zernike-corrected dataset, a significant lateral shift of approximately $1 \mu m$ between the two images can be observed. This is empircally compatible with the fact that the wavefront corrected with the shift-less base presented a tip-tilt component of $1.2 \lambda$ peak to valley, corresponding to a lateral shift of $1.2$ Airy units in the focal plane, which for the reported setup is equivalent to approximately $1 \mu m$. Panel C shows an $x-z$ plane within the dataset, where the Zernike-corrected dataset was extended in the axial direction for clarity. The data clearly shows how the shift-less base corrected the mild spherical aberration, making the PSF brighter and axially shorter, while the Zernike-based correction introduced a significant deformation on the PSF and significantly shifted the imaging plane. Panel D shows the same small bead in the three datasets. The resolution of the datasets was measured as the full width at half maximum of the detail. The uncorrected resolution was $710 \pm 2\: nm$. While the Zernike based correction improved the resolution to $600 \pm 2\: nm$, using the shift-less base for correction reduced it further to $540 \pm 2\: nm$.
Fig. 3. Microbeads experiment: A) Uncorrected, shift-less corrected and Zernike corrected images in the focal plane. Scale bar is $3 \: \mu m$. B) Images of the planes with highest correlation with the uncorrected focal plane in the three three-dimensional datasets. C) Example x-z slices from the datasets, red line represents the microscope’s focal plane. Scale bar is $1 \: \mu m$. D) Maximum intensity projections of a sub-diffraction object in the field of view. Scale bar is $200 \: nm$. E) x-z and y-z maximum intensity projections of the object in D. Vertical scale bar is $500 \: nm$, horizontal scale bar is $200 \: nm$ F) schematic representation of the position of the focal planes in B. G) Phase correction after optimization with the shift-less base. H) Phase correction after optimization with the Zernike base.
Fig. 4. In-vivo experiments: Images of fluorescence from sRhoVR-1 injection in the barrel cortex of anesthetized mouse. Images are reported in the absence of correction, in the case of correction with the shift-less base, and of correction with the Zernike base. A) Two-dimensional image at the focal plane of the microscope. Scale bar is $20 \: \mu m$. B) Detail from the red highlighted area of panel A, both in x-y and x-z projection. Scale bars are $10 \: \mu m$. C) Detail from the magenta highlighted area of panel A, both in x-y and x-z projection. Scale bars are $10 \: \mu$.D) Phase correction after optimization.
Experiments in a more realistic scenario for adaptive optics in two-photon microscopy were performed in-vivo in an anesthetized mouse. Datasets were acquired with $1$ by $1$ by $1\: \mu m^3$ voxels, a relatively lower sampling compared to the microbeads measurements due to the necessity of minimizing the experimental time for the animal due to ethical necessity. Datasets were acquired over a $200$ by $200$ by $50\: \mu m^3$ field of view. Results are reported in Fig. 4. While, due to the nature of the experiment, small motion artefacts are present in the images, evident shifts can be observed in the Zernike-corrected dataset when compared to the uncorrected and shift-less corrected datasets. In this case, while an increase in resolution can be observed after correction with the shift-less base, it is quite evident how correction with the Zernike base, due to the sparsity of the distribution of the fluorophore, actually slightly impaired the resolution of the system in order to shift the image focus on brighter details of the three-dimensional image. This can be easily observed, for example, in the two cell bodies visible in the y-z projection in panel C, where the shift-less base correction allowed to visibly distinguish the two bodies, while after Zernike correction they appear as a single mass.
4. Discussion
In this manuscript, we described and tested a procedure for calibration of a shift-less aberration base for sensorless adaptive optics in nonlinear microscopy applications. The described method requires some modifications of a nonlinear microscopy setup which can be converted into an adaptive setup with a relatively low cost. Most importantly, its implementation is extremely simple and is within the reach of research groups with expertise in the development of nonlinear optical setups.
As discussed in the introduction, shifts in the microscope’s field of view position during adaptive optics correction are generally considered negligible. However, in samples presenting sparse and particularly bright objects throughout their three-dimensional field of view, such as those imaged in the results section of this manuscript, a generic image metric may reach higher values when applying aberrations shifting the field of view towards brighter features.
In addition, it is increasingly popular in microscopy applications of adaptive optics to independently correct tiled regions of the field of view. In this case, even in the presence of an optimal correction, relative shifts in the field of view of the image tiles can lead to artefacts in the full reconstructed image.
Due to the simple implementation of the calibration procedure, as well as the potential benefits in its use, we would recommend the use of a calibrated shift-less base in any sensorless adaptive optics system for nonlinear microscopy methods.
Funding
Horizon 2020 Framework Programme (785907); Ministero dell’Istruzione, dell’Università e della Ricerca (DM MIUR 10.01.2018); Regione Lombardia (POR FESR 2014-2020); European Research Council (101016923).
Acknowledgments
J.M. and P.P. acknowledge funding from the Horizon 2020 Framework Programme under grant agreement No. 785907 and from the Ministero dell’Istruzione, dell’Université della Ricerca "Dipartimenti di eccellenza 2018-2022" grant agreement No. DM MIUR 10.01.2018 to the Department of Biomedical, Metabolic and Neural Sciences. D.P. and B.T. acknowledge funding from the European Union project CRIMSON under grant agreement No. 101016923 and from the Regione Lombardia project NEWMED under grant agreement No. POR FESR 2014-2020. The authors would like to thank the group of Andrea Alessandrini at the Physics Department of the university of Modena and Reggio Emilia for providing custom machined parts for the experimental setup. The authors also acknowledge the help of Riccardo Scorrano from Politecnico’s Physics department workshop for assistance in design and manufacturing of 3D printed parts.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. M. J. Booth, “Adaptive optical microscopy: the ongoing quest for a perfect image,” Light: Sci. Appl. 3(4), e165 (2014). [CrossRef]
2. P. Pozzi, D. Gandolfi, C. A. Porro, A. Bigiani, and J. Mapelli, “Scattering compensation for deep brain microscopy: The long road to get proper images,” Front. Phys. 8, 26 (2020). [CrossRef]
3. J.-W. Cha, J. Ballesta, and P. T. So, “Shack-hartmann wavefront-sensor-based adaptive optics system for multiphoton microscopy,” J. Biomed. Opt. 15(4), 046022 (2010). [CrossRef]
4. M. J. Booth, “Wavefront sensorless adaptive optics for large aberrations,” Opt. Lett. 32(1), 5–7 (2007). [CrossRef]
5. J. M. Bueno, E. J. Gualda, and P. Artal, “Adaptive optics multiphoton microscopy to study ex vivo ocular tissues,” J. Biomed. Opt. 15(6), 066004 (2010). [CrossRef]
6. P. Pozzi, O. Soloviev, D. Wilding, G. Vdovin, and M. Verhaegen, “Optimal model-based sensorless adaptive optics for epifluorescence microscopy,” PLoS One 13(3), e0194523 (2018). [CrossRef]
7. P. Pozzi, D. Wilding, O. Soloviev, H. Verstraete, L. Bliek, G. Vdovin, and M. Verhaegen, “High speed wavefront sensorless aberration correction in digital micromirror based confocal microscopy,” Opt. Express 25(2), 949–959 (2017). [CrossRef]
8. D. Saha, U. Schmidt, Q. Zhang, A. Barbotin, Q. Hu, N. Ji, M. J. Booth, M. Weigert, and E. W. Myers, “Practical sensorless aberration estimation for 3d microscopy with deep learning,” Opt. Express 28(20), 29044–29053 (2020). [CrossRef]
9. M. J. Booth, D. Débarre, and T. Wilson, “Image-based wavefront sensorless adaptive optics,” in Advanced Wavefront Control: Methods, Devices, and Applications V, vol. 6711 (International Society for Optics and Photonics, 2007), p. 671102.
10. H. Yang, O. Soloviev, and M. Verhaegen, “Model-based wavefront sensorless adaptive optics system for large aberrations and extended objects,” Opt. Express 23(19), 24587–24601 (2015). [CrossRef]
11. H. Lin, H. J. Lee, N. Tague, J.-B. Lugagne, C. Zong, F. Deng, J. Shin, L. Tian, W. Wong, M. J. Dunlop, and J. X. Cheng, “Microsecond fingerprint stimulated raman spectroscopic imaging by ultrafast tuning and spatial-spectral learning,” Nat. Commun. 12(1), 3052 (2021). [CrossRef]
12. D. G. Ouzounov, T. Wang, M. Wang, D. D. Feng, N. G. Horton, J. C. Cruz-Hernández, Y.-T. Cheng, J. Reimer, A. S. Tolias, N. Nishimura, and C. Xu, “In vivo three-photon imaging of activity of gcamp6-labeled neurons deep in intact mouse brain,” Nat. Methods 14(4), 388–390 (2017). [CrossRef]
13. P. Pozzi, C. Smith, E. Carroll, D. Wilding, O. Soloviev, M. Booth, G. Vdovin, and M. Verhaegen, “Anisoplanatic adaptive optics in parallelized laser scanning microscopy,” Opt. Express 28(10), 14222–14236 (2020). [CrossRef]
14. P. Rajaeipour, A. Dorn, K. Banerjee, H. Zappe, and Ç. Ataman, “Extended field-of-view adaptive optics in microscopy via numerical field segmentation,” Appl. Opt. 59(12), 3784–3791 (2020). [CrossRef]
15. T.-L. Liu, S. Upadhyayula, D. E. Milkie, V. Singh, K. Wang, I. A. Swinburne, K. R. Mosaliganti, Z. M. Collins, T. W. Hiscock, J. Shea, A. Q. Kohrman, T. N. Medwig, D. Dambournet, R. Forster, B. Cunniff, Y. Ruan, H. Yashiro, S. Scholpp, E. M. Meyerowitz, D. Hockemeyer, D. G. Drubin, B. L. Martin, D. Q. Matus, M. Koyama, S. G. Megason, T. Kirchhausen, and E. Betzig, “Observing the cell in its native state: Imaging subcellular dynamics in multicellular organisms,” Science 360(6386), eaaq1392 (2018). [CrossRef]
16. J.-H. Park, L. Kong, Y. Zhou, and M. Cui, “Large-field-of-view imaging by multi-pupil adaptive optics,” Nat. Methods 14(6), 581–583 (2017). [CrossRef]
17. D. Ancora, T. Furieri, S. Bonora, and A. Bassi, “Spinning pupil aberration measurement for anisoplanatic deconvolution,” Opt. Lett. 46(12), 2884–2887 (2021). [CrossRef]
18. P. Pozzi, D. Gandolfi, M. Tognolina, G. Chirico, J. Mapelli, and E. D’Angelo, “High-throughput spatial light modulation two-photon microscopy for fast functional imaging,” Neurophotonics 2(1), 015005 (2015). [CrossRef]
19. V. N. Mahajan, “Zernike circle polynomials and optical aberrations of systems with circular pupils,” Appl. Opt. 33(34), 8121–8124 (1994). [CrossRef]
20. R. U. Kulkarni, M. Vandenberghe, M. Thunemann, F. James, O. A. Andreassen, S. Djurovic, A. Devor, and E. W. Miller, “In vivo two-photon voltage imaging with sulfonated rhodamine dyes,” ACS Cent. Sci. 4(10), 1371–1378 (2018). [CrossRef]
