## Abstract

It has been demonstrated by several authors that the optical turbulence parameters associated with a given adaptive optics (AO) run—the seeing angle and outer scale—can be determined from a statistical analysis of the commands of the system’s deformable mirror (DM). The higher the accuracy on these parameters, the more we can make use of them, allowing for instance a better estimation of the seeing statistics at the telescope location or a more accurate assessment of the performance of the AO system. In the context of a point spread function reconstruction project (PSF-R) for the W. M. Keck observatory AO system, we decided to identify, in the most exhaustive way, all the sources of systematic and random errors affecting the determination of the seeing angle and outer scale from the DM telemetry, and find ways to compensate/mitigate these errors to keep them under 10%. The seeing estimated using our improved DM-seeing method was compared with more than 70 nearly simultaneous seeing measurements from open-loop PSFs on the same optical axis, and with independent seeing-monitor measurements acquired at the same time but far from the telescope (DIMM/MASS): the correlation with the open-loop PSF is very good (the error is about 10%), validating the DM-seeing method for accurate seeing determination, while it is weak and sometimes completely uncorrelated with the DIMM/MASS seeing monitor data. We concluded that DM-based seeing can be very accurate if all the error terms are considered in the DM data processing, but that seeing taken from non-collocated seeing monitors is of no use even when moderate accuracy is required.

© 2018 Optical Society of America

## 1. INTRODUCTION

We revisit in this paper the method to retrieve, from the commands of the
deformable mirror (DM) of an adaptive optics (AO) system, two of the most
important parameters of optical turbulence (OT): (1) the seeing
angle, defined as the angular full-width-at-half-maximum (FWHM) of the
seeing-limited long-exposure PSF and noted ${w}_{0}$ here, or equivalently the Fried parameter
${r}_{0}$; and (2) the optical outer scale,
noted ${L}_{0}$. The method was introduced by Rigaut
*et al.* [1],
and has since been implemented by many other groups (see references), and
we are using it in the context of a point spread function reconstruction
(PSF-R) project for the W. M. Keck observatory AO system (a technique
pioneered by Véran *et al.* [2]). The importance of a highly accurate
seeing determination for PSF-R motivated the research described in this
paper.

In the literature, the DM-based seeing monitor is mentioned as a
particularly simple and efficient method for analysis such as AO system
performance evaluation, assessment of the current seeing conditions, and
PSF-R. Indeed, by using the AO data itself, it is possible to get an
estimate of the seeing obtained *at* the telescope and
*during* the observation—something not provided by
seeing monitors because they are generally *not collocated*
with the telescope, nor by measuring the seeing
*before/after* the AO run, as the seeing changes too
rapidly.

In the specific case of PSF-R, the seeing angle is critically needed to
reconstruct the component of the PSF associated with the high spatial
frequencies of the turbulent aberration (the DM *fitting
error* in the AO jargon) located beyond the cutoff spatial
frequency of the AO system, 1/(2 DM pitch). This error indeed dominates
the residual wavefront in the case of good seeing conditions, and is
always significant in other cases.

Let us consider the case of the W. M. Keck 10 m telescope AO system
(DM pitch 0.5625 m in the pupil), observing at
$\lambda =1.2\text{\hspace{0.17em}}\mathrm{\mu m}$ with a seeing of ${0.8}^{\prime \prime}$. We find (not showing the details) that with
a relative error of 10% on the seeing determination, the error on the
Strehl is 10% as well. For a larger seeing angle, the error would be
proportionally higher. For AO data processing (photometry, astrometry,
deconvolution, etc.), this might be unacceptably large, knowing that the
other PSF metrics errors, i.e., width or integrated energy, would be in
the same proportions (see for instance Lu *et al.*
[3]).

An accurate and simultaneous determination of the outer scale
${L}_{0}$ is also very important: because
${L}_{0}$ decreases the amplitude of the turbulent
wavefront, it makes the angular size of a point-like object imaged through
OT smaller than what it would be with a fully developed Kolmogorov
turbulent flow. A small ${L}_{0}$ therefore has the same overall effect as a
small seeing angle. Over- or underestimating the effect of the outer scale
would produce an incorrect seeing angle estimation, with the consequences
we have just seen. Note that there is no correlation between
${w}_{0}$ and ${L}_{0}$. Indeed, the widely used von Karman model is
only an *empirical* method of correcting the infinite
scales Kolmogorov model to account for the fact that there is
*necessarily* an external limit to the size of the
turbulent flow, with the consequence that the effect of the OT is always
less severe than what is predicted by the Kolmogorov theory. For this
reason in particular, the FWHM of an experimental long-exposure
seeing-limited PSF will always be smaller than the
${w}_{0}$ predicted for a fully developed Kolmogorov
flow.

Therefore, for the PSF-R project, we needed to identify as much as possible
all sources of *systematic* and *random*
errors on the seeing and the outer scale determination, known or unknown
from the literature, and find ways to mitigate or suppress them. In this
paper, we describe these errors and the way they are considered in our
DM-seeing monitor algorithm, as well as the results of 841 AO testing runs
from 2011 to 2017 in natural and laser guide star (NGS/LGS) modes. Our
model considers in detail the following components of the problem in order
of importance (Keck-II AO is a Shack–Hartmann wavefront sensor
based system):

- 1. Choice of the pupil area to avoid the edges effect,
- 2. Impact of the central obscuration,
- 3. Zernike polynomial projection error and its mitigation by an adaptation of the polynomials to the DM influence function basis,
- 4. Zernike polynomial undersampling,
- 5. Deformable mirror influence function model,
- 6. Bad and good Zernike modes,
- 7. Contribution of the outer scale,
- 8. Wavefront sensor noise,
- 9. Shack–Hartmann wavefront sensor spatial aliasing,
- 10. Guide star mode, natural or laser.

Our seeing estimation method has been validated with a comparison of
*almost* simultaneous seeing measurements from open-loop
PSF. This is presented in the second part of this paper, where we also
compare the DM-seeing results with simultaneous measurements from seeing
monitors not collocated with the telescope, and show that the correlation
is weak at best.

#### A. Previous Research

Our study complements a previous study by Schöck
*et al.* [4], who used the same kind of method on the same telescope and
AO system, but on open-loop wavefront sensor (WFS) data. Here, we use
closed-loop DM data: this has the significant advantage that there is
no need to open the loop to acquire the seeing data; therefore, the
seeing estimation is absolutely simultaneous with the AO run,
something mandatory for PSF-R.

Many authors have already used the DM-seeing method on several AO
systems, essentially for AO system performance diagnostics during
commissioning. Rigaut *et al.* [1] were probably the first to use the
method for AO performance check, on the system COME-ON; Véran
*et al.* [2] used the method in the context of the development of the
first PSF-R algorithm, on the curvature-sensing-based AO system PUEO
on the CFHT; Fusco *et al.* [5] have used the method on NAOS, the
VLT first AO system, and did a first analysis of the error terms very
similar to ours; the latter analysis was complemented by an analysis
of the WFS aliasing effect by Kolb *et al.*
[6].

Our own work considers the same terms as the previous authors, as well as additional terms which were not considered before. We have put a particular effort into the construction of a modal basis adapted to the telescope pupil, using a very accurate influence function empirical model, and to the selection of the modes actually used for the OT parameter estimation. Our outer-scale damping model has a larger domain of validity than previous models and is adapted to the scale of a 10 m telescope. In short, what makes this paper original is the emphasis put on details not considered before and the use of highly accurate models to account for complex effects. The large amount of sky data and the comparison with open-loop and seeing-monitor data allows us to clearly demonstrate the validity of the procedure and provide an estimation of the seeing determination accuracy.

## 2. DEFINITIONS

Let us recall the definitions of the OT quantities of interest in this study. For an introduction to OT theory, see Tatarski [7] and Roddier [8].

#### A. Seeing Angle

The seeing angle, noted as ${w}_{0}$, is defined as the FWHM of the long-exposure image of a point source seen through OT (ignoring diffraction from the telescope) when the turbulent flow is fully developed, i.e., the turbulent flow structure follows a Kolmogorov law, with infinite outer scale and null inner scale.

#### B. Fried Parameter

${w}_{0}$ is related to the Fried parameter ${r}_{0}$ (Fried [9]), defined as the diameter of a telescope having the same optical resolution (integral of the OTF) as the long exposure optical resolution when looking through the aberrated optical system formed by OT. ${r}_{0}$ depends on the turbulence strength and the imaging wavelength $\lambda $, according to

#### C. Optical Outer Scale

OT is a consequence of the turbulent mixing, due to the wind, of air masses at different temperature and hence different refractive index. Let us consider the typical difference $\mathrm{\Delta}n=n(\mathbf{r}+\mathrm{\Delta}\mathbf{r})-n(\mathbf{r})$ of the refractive index between two points in the turbulent flow. Because of the turbulent mixing, the larger the separation, the higher the probability of experiencing large refractive index differences between the two points. If there is no upper limit to the extension of the turbulent flow, $\mathrm{\Delta}n$ will simply increase with $\mathrm{\Delta}r$, and it is shown in Tatarski [7] that on average $\u27e8\mathrm{\Delta}{n}^{2}\u27e9\sim \mathrm{\Delta}{r}^{2/3}$.

Now, as there is necessarily an upper limit to the size of the largest
eddies, the occurrence of large $\mathrm{\Delta}n$ is less probable when
$\mathrm{\Delta}r$ increases beyond a certain limit, and
$\u27e8\mathrm{\Delta}{n}^{2}\u27e9$ saturates to a constant for large
separations. The saturation distance is characterized by the so-called
outer scale of optical turbulence ${L}_{0}$, related to the size of the largest
eddies in the flow. Measurements in turbulent flow by many authors
have clearly demonstrated the existence of this optical outer scale
(see for instance Conan *et al.* [11] and Ziad
*et al.* [12]).

As a consequence, the larger the telescope diameter with respect to ${L}_{0}$, the lower the wavefront RMS with respect to what we would expect for an infinite flow. Now, because ${r}_{0}$ is defined for an infinite flow, it is therefore mandatory, when using the DM-seeing method, to evaluate simultaneously the outer scale with the seeing angle in order to make an independent determination of the latter. Because a finite outer scale improves the image quality with respect to the infinite case, not taking into account ${L}_{0}$ would lead to overestimation of the Fried parameter, and underestimation of the seeing. The empirical outer-scale attenuation model we have developed for this purpose is described later in the text.

## 3. SUMMARY OF THE DM-SEEING METHOD

The AO system compensates the OT wavefront error by using the DM. As a
consequence, at each instant, the DM surface (times 2, and introducing a
cosine factor to take into account possible non-normal incidence) is
naturally a good representation of the turbulent wavefront seen by the AO
system up to its cutoff spatial frequency, with some perturbations due to
the classical low-order AO errors, i.e., the servo-lag, WFS noise, and WFS
aliasing. Therefore, from the DM command statistics, it must be possible
to get an estimate of the average seeing associated with the AO
observation. This idea has been validated in the context of PSF-R by
Véran *et al.* [2], on the curvature wavefront-based AO system PUEO
(Canada-France-Hawaii Telescope).

The Zernike polynomials basis is generally used for this purpose, and is
also our choice here. A Karhunen Loeve basis could be considered too, but
we would lose the benefit of having an analytical expression for both the
polynomials and the optical turbulence statistics (Noll distribution). It
is a *modal* representation, as opposed to the DM influence
function representation, which is *zonal*. The expression
for the covariance of the Zernike coefficients ${z}_{k}$ and ${z}_{l}$ in the case of a pure Kolmogorov turbulent
flow is given by Noll [13]. Winker
[14] has supplemented Noll’s
model with the damping effect of the outer scale, and we have

In AO systems, the DM commands are generally not expressed in the Zernike basis, but rather in a basis appropriate to the system’s modes or in the DM influence function basis. In both cases, the Zernike covariance matrix is obtained from

As given, however, this model cannot be used because many effects (listed in the introduction) generating systematic and random errors affecting the DM commands and the Zernike basis construction have not been considered. The objective of this paper is specifically to address all these errors terms, and show how they can be compensated.

## 4. CONSTRUCTION OF A MODAL BASIS, CONSIDERING THE PUPIL EDGES AND THE DM ZONAL MODES

In this section, we show how the classical Zernike modal basis, as defined by Noll [13], and the coefficients covariance model ${\mathcal{N}}_{k,l}$ are modified to suppress or minimize the impact of the following effects: the pupil edge, the central obscuration, the Zernike projection error and Zernike under-sampling, the DM influence functions model, and the bad modes.

#### A. Pupil Edge Effects

In an AO system, actuators whose influence functions are partially
outside of the *external* pupil edge have to be given a
larger command than the ones that are fully illuminated. Indeed, those
actuators, if given a command amplitude identical to the illuminated
ones, generate a DM surface that is too low in amplitude because at
this location there are less actuators whose influence functions can
overlap. Indeed, the total surface amplitude of a set of (for
instance) 3-by-3 neighbor actuators is higher than a single actuator
receiving the same command, because the wings of the 3-by-3 influence
functions add together and lift the DM surface to a higher amplitude.
To compensate for the lack of amplitude inside the pupil, on the edge,
the AO control simply gives a higher command to the corresponding
actuators.

Other issues affect the actuator commands, on the pupil external edge as well as inside the central obscuration: at this location, at least in the case of a SH-WFS, the lenslets are vignetted, generating a systematic error as well as a random error on the wavefront measurement due to the lower signal-to-noise ratio. On top of this, in the specific case of the Keck hexagonal pupil, because the latter rotates with respect to the lenslet array, the vignetting changes during the exposure, as we understand from Fig. 1.

All these effects make the DM surface statistics not stationary along the pupil external/internal edges. Figure 2 shows the standard deviation of the DM surface of the Keck-II AO system for one of our on-sky PSF-R experiments. It is constructed as the RMS value of the DM surface at each DM points for 35,912 loop sequences at a frame rate of 1 kHz. We can see that the surface deviation is 2–4 time larger, on the external edges of the DM as well as inside the central obscuration, than inside the fully illuminated pupil.

Now, with the DM seeing monitor technique we are using, it is assumed
that the turbulent wavefront *is* stationary, within a
*circular* pupil. So, to avoid overestimating the
turbulent wavefront RMS and the seeing angle, and to have a circular
pupil (in the specific case of Keck), we must limit the seeing
estimation within a smaller annular section on the telescope pupil
where the DM commands can be considered stationary.

Figure 1, for Keck, shows the circular area on the 36-segment pupil which is always illuminated regardless of the pupil orientation with respect to the WFS lenslet array. It is therefore clear that this limit (9 m in diameter) is a good choice to define the pupil for the seeing computation (the telescope central obscuration is 2.65 m). Figure 2 clearly shows that the DM surface is stationary within this 9/2.65 m pupil (white circles), at least relative to the edges.

In practice, to decide on the pupil diameter for the seeing estimation,
we can compute how stationary the DM surface RMS is when averaged over
an annulus of increased external radius, from the central obscuration
to the telescope pupil radius. We did this with the Keck pupil for a
radius from a little more than the central obscuration radius up to
the maximum telescope *hexagonal* pupil radius,
6.2 m. The result, for 79 AO runs, is shown in
Fig. 3.

As we can see, there is some level of non-stationarity in the fully illuminated area (otherwise the RMS of the DM wavefront RMS would be 0), but this is in general relatively stable until the average hexagonal pupil radius is reached near 5 m. Beyond, the non-stationarity increases very strongly. It therefore seems very reasonable to consider the pupil 9/2.56 meters for the seeing estimation.

Note that the secondary support structure arms are very thin (2.54 cm) in the case of the Keck telescopes, and therefore we do not take this structure into account in the pupil model. Only the very high-order Zernike polynomials (radial order 600) would be affected at this spatial scale.

#### B. Modified Zernike Basis

The DM commands are applied to the actuators, so the actual DM surface shape belongs to the space defined by the actuators’ influence function basis. Noll’s model, on the other hand, is given in the Zernike basis, so a change of basis is required which is very simple to do, numerically, using an orthogonal projection of the Zernike basis into the influence functions basis. However, this procedure introduces systematic errors.

First, the Zernike polynomials are *not* well
represented by the influence function basis. The difference between a
Zernike polynomial and its projection on the DM basis can be
significant, more than a few percent, and varies in a complicated way
with the Zernike index $j$ and azimuthal order
$m$, as Fig. 4 shows. This would generate a
*systematic* error in the estimation of the variances
of the DM commands expressed in the Zernike basis simply because the
transformation matrix does not generate the true Zernike
coefficients.

The second difficulty comes from the limited numerical sampling of the influence functions and Zernike polynomials while computing the DM to Zernike transformation matrices. This error term is shown by the black curves in Fig. 4. Thus, even without the projection error, the limited numerical sampling already induces a systematic error of a few percent, strongly depending on the Zernike index.

The third source of error comes from neglecting the central obscuration. When it is relatively large (say, more than 10% of the pupil diameter), the central hole significantly changes the norm of some Zernike polynomials and generates, if not taken into account, a systematic error in the coefficient variances, as illustrated in Fig. 5.

The solution to these systematic errors is simple: orthogonality of the
modes that we use to express the OT wavefront error is
*not* required—independence of the modes is
indeed sufficient—so we can simply consider, as a modal basis,
the projection of the Zernike polynomials on the influence functions
basis, including the central obscuration. The sampling error can be
minimized by using the highest sampling resolution compatible with a
reasonable computation time. We shall call these polynomials the
*modified Zernike polynomials*, noted
${\widehat{Z}}_{j}$. They differ from the true Zernike by a
term ${e}_{j}$,

### 1. Realistic Empirical Model for the DM Influence Function

In order not to add another source of systematic error to the seeing estimation, it is clear that the influence function model to use in the preceding basis transformation has to be as realistic as possible; otherwise it would make no sense to consider the projection error, as it depends on the influence function shape. For the Keck AO DM, Gaussian, double-Gaussian, or/and Moffat profiles are not accurate enough because they do not capture the variation of the influence function section shape, from square (bottom) to circular (top).

We had access to accurate interferometric measurements of the influence functions of the Gemini North AO system DM (Altair), made from the same technology and produced by the same manufacturer as Keck’s DM (Xinetics, Inc.) From these measurements, we have built a high-fidelity empirical model, reproducing the influence function section shape at all heights.

Our model, $\mathcal{I}(x,y)$, is given below and is represented in Fig. 6. The unit of the coordinate $(\mathbf{x},\mathbf{y})$ is pitch length. It is essentially a negative exponential model, with an adjustment to account for the negative drop at the foot of the influence function, in which the section of the influence function varies gradually from a circle at its top to a square with round edges on its bottom (modeled with the functions $\rho $ and $\epsilon $). There is an overall mask function $\mathcal{M}$ to strongly but smoothly cut the edge of the influence function to limit its extension over the DM:

#### C. Constructing the Modified Zernike Basis Coefficient Variance Model for Kolmogorov Turbulence

Of course, with the Zernike basis modification, *we
cannot* use Noll’s covariance model anymore, and it has
to be recomputed. This is easily done with a Monte Carlo simulation of
a large number of independent turbulent phase screens (10,000 screens
allows for 1% convergence) with $D/{r}_{0}=1$ and ${L}_{0}=\infty $, projected orthogonally on the modified
Zernike basis. From the resulting series of modified Zernike
coefficients, the covariances are directly computed. Note that we
assume that Winker’s correction factor is only marginally
affected by the systematic errors indicated previously, so only the
term ${\mathcal{N}}_{k,l}$ is modified. Indeed, an exact calculation
of the correction factors would require us to use the modified Zernike
polynomials, but this would only change the Zernike variances from
Winker’s model by the same amount that we have seen above for
Noll’s variance, i.e., a few percent. In other words, we would
change the *correction* factor by a few percent, and
the effect would therefore be minimal. Therefore, we have decided not
to recompute Winker’s model.

#### D. How Many Modes Should Be Considered and Which Ones Should Be Avoided?

The higher the Zernike polynomial index, the higher the spatial frequencies within the polynomial. Now, beyond some j-index, the oscillation period in the radial and azimuthal components of the polynomial will be much shorter than the influence function spatial extension, and the projection coefficient ${b}_{j,i}$ in Eq. (8) will gradually drops to zero.

For the Keck system, using our influence function model, we have found that above $j=2000$, the projection coefficient ${b}_{j,i}$ becomes negligible and the DM is no longer sensitive to these polynomials. But in the Keck case, as we will see, the Zernike polynomial sampling error just introduced affects the variance model for j-indexes much lower than $j=2000$.

Besides, there is no need to use such a large number of modes; indeed, a few hundred is enough to get an excellent adjustment of the modified Noll–Winker variance model. And finally, it is intuitive that for the high-order modes, the impact of the wavefront sensing noise can only increase. In the end, only a careful examination of the signal-to-noise ratio of the AO system considered and the effect of the numerical undersampling can tell what must be the maximum number of modes to use.

Finally, the modes susceptible of being affected by
*non*-turbulent sources of aberrations—the bad
modes—must be excluded from the seeing computation because
their coefficients will carry an *excess* of variance
relative to the turbulent variance. Sources of dynamic non-turbulent
errors are generally the wind-induced telescope jitter, telescope
tracking errors, and mirror wrapping in the case of large or segmented
primary mirrors generating astigmatism, defocus, and possibly some
coma. These instrumental aberrations are low order, and are
practically null at orders higher than, generally, spherical
aberration (fourth radial order). This excess of low-order variance
would be interpreted as a larger outer scale than the actual value,
and as we are correcting the Zernike variances for the damping effect
of the outer scale in order to get an estimation of the Fried
parameter, sub-correcting would lead to an underestimation of
${w}_{0}$. Of course, only a quantitative
estimation of the instrumental variance for each mode can tell which
are the modes to avoid, but clearly at least tip-tilt must be
excluded.

#### E. Modified Zernike Coefficients Variance in Practice: The Case of Keck AO

We have applied the Monte Carlo method just described to the Keck AO system influence function basis and pupil. Figure 7 shows the variance of the modified Zernike coefficients for a turbulence $D/{r}_{0}=1$, an infinite outer scale, a central obscuration $\epsilon =2.65/9=0.294$, and a resolution of 29 pixels per actuator pitch.

As said previously, we exclude tip-tilt from the basis, but also the defocus and the two astigmatisms: indeed, even if we are not certain that the Keck telescope optical system generates these second-order aberrations, these are very often seen in other systems, so we prudently prefer to discard them. To give an order of magnitude of the impact of an excess of defocus and astigmatism, we ran our seeing and outer-scale estimation method on a simulated distribution of Zernike variances for a seeing of ${1}^{\prime \prime}$ and an outer scale of 13 m (close to the average value we found for the telescope). If each of the ${Z}_{4}$, ${Z}_{5}$, and ${Z}_{6}$ have an extra instrumental error of 100 nm RMS, the seeing angle is overestimated by 2%. If we double the instrumentation error, the seeing angle overestimation is 4%. The effect is not enormous, but it can be easily avoided by unselecting these potentially instrumental-error-sensitive aberrations.

The first modified Zernike modes we are therefore considering in the
model are the two comas ${Z}_{7}$ and ${Z}_{8}$, and the last are the ones associated
with the radial order $n=17$, corresponding to
${Z}_{171}$. The effect of the central obscuration is
well apparent: for the azimuthal orders $m=0$ and $m=1$, the modal variance is several orders of
magnitude higher than Noll’s model, showing *a
posteriori* the importance of considering this parameter. For
most modes, though, the variance is close to Noll’s model.

In the same figure, we can also see that beyond the index $j=153$, corresponding to the last polynomial of radial order $n=16$, the departure with Noll’s variance becomes erratic. Because this is not the case for the previous modes, we think this is a consequence of the numerical undersampling error. We decided therefore to limit the Zernike index range from 7 to 153 (radial orders 3 to 16), which still makes 147 good modes.

## 5. OUTER-SCALE VARIANCE ATTENUATION MODEL

We now need a model for ${\mathcal{W}}_{k,l}$, the damping factor of the modal variances due to the optical outer scale.

Winker [14] has developed the analytical expressions for the covariance of the Zernike coefficients, in a Von Karman model of the optical turbulence phase spatial power spectrum, as a function of $D/{L}_{0}$. These equations contain hypergeometric functions which are not practical to handle in a minimization algorithm such as ours. Also, we only need a model for the variances of the Zernike coefficients, not the covariances, so in our case $k=l$.

Winker provided us with a grid of ${\mathcal{W}}_{j,j}$ values for a range $D/{L}_{0}=[0.002,2]$ (or ${L}_{0}/D=[0.5,500]$) and Zernike indexes up to $j=153$ (or $n=16$), and from these data we have built an empiric model of the damping factor (see Fig. 8). In its domain of validity, the accuracy of our model is about 1%.

Defining $x\equiv D/{L}_{0}$, with $n$ being the radial order associated with the Zernike index $j$, the model is given by

Figure 9 shows the variance attenuation expected for a 9 m pupil and an outer scale of 13 m, equal to the average of what we have measured on the Keck telescope (see Section 7). It can be seen that when the outer scale is on the order of the pupil diameter, its effect on the variance is very significant even on the highest radial orders.

## 6. PREPARATION OF THE DM DATA

The DM commands ${m}_{k}$ are built from the application of the command matrix to the WFS measurements vector, followed by the application of a control law (at the minimum a simple integrator). For a well adjusted and aligned system, the sources of WFS measurement error are essentially the detector and photon noise, and the WFS spatial aliasing of the high-frequency part of the turbulent wavefront. These errors are added to the DM commands during reconstruction and control.

It is relatively easy to build a model of the covariances associated with
these two errors terms, allowing us to identify and remove them from the
DM command statistics. This procedure was first described by Véran
*et al.* [2]
for a *modal* system and a curvature WFS. Here it is
adapted to the Keck AO, a zonal and Shack–Hartmann WFS based
system.

Assuming that the AO system does not suffer from significant calibration and alignment errors, the command ${m}_{k}$ to actuator $k$, at each loop instant, is given by the sum of the following components:

where ${a}_{\parallel ,k}$ is the projection coefficient on the $k$th influence function of the low-frequency part of the turbulent wavefront that can be seen by the WFS, averaged over the WFS integration time and delayed by the data processing time required by the loop operations; ${n}_{k}$ is the random command noise associated with the WFS noise; and ${a}_{\parallel ,k}^{\perp}$ is the command error due to the WFS spatial aliasing of the high spatial frequencies into the low-frequency domain.The covariance of the coefficients ${a}_{\parallel ,k}$ is what is needed in Eq. (4) to compute the modified Zernike coefficients covariance, on which the modified Noll–Winker model is applied. It is given by

Let us discuss first how to evaluate the noise covariance from the DM commands themselves. We will discuss the estimation and compensation of the aliasing contribution next.

#### A. Compensating for the Wavefront Sensor Noise

We are aware of two methods to evaluate the noise level. The first
involves a statistical modeling of the WFS signal: comparing the real
statistics with the expected statistics in the absence of noise, the
noise variance can be estimated. This powerful technique has been
developed by Véran *et al.* [2] and tested on a curvature-sensing
WFS. An adaptation to the Shack–Hartmann WFS is given in
Jolissaint *et al.* [15]. This technique requires the raw WFS data (the
WFS camera pixel values). We did not get access to these data in the
Keck experiment, so we had to turn to the second technique: the noise
power spectrum method, detailed in the following.

### 1. Estimating the WFS Noise Variance Using the Noise Transfer Function

Figure 10 shows the temporal power spectrum (t-PSD) of the command to one of the actuators of the Keck AO system DM. At low frequency, the spectrum is dominated by the atmospheric signal. At high frequency, the spectrum saturates to a noise plateau. As the noise transfer function can be computed from the parameters of the AO loop, which are known by design, it is possible to adjust to the data a model of the filtered t-PSD of the two components $\mathbf{m}$ and $\mathbf{n}$, and extract the noise variance from the fit (the procedure to go from the noise t-PSD to the noise variance will be detailed later).

Let us develop the DM commands t-PSD model. The AO loop diagram is shown in Fig. 11. ${\mathbf{a}}_{\parallel}$ indicates the vector of the projection of the turbulent phase into the DM mode basis (any of the actuators), and $\mathbf{e}$ is the residual after partial correction by the DM command $\mathbf{m}$, i.e., $\mathbf{e}={\mathbf{a}}_{\parallel}-\mathbf{m}$. The model is developed in the continuous frequency domain ($\nu $ [Hz]), and hence the Laplace transform is used. Once the AO system is running, the transient response is quickly over, and therefore the argument of the Laplace transform can be replaced with $s=i2\pi \nu $.

With ${\nu}_{s}$ standing for the loop sampling frequency, the transfer functions of each block are as follows:

- 2. The WFS aliasing transfer function is the same as the WFS transfer function; it is only the shape of the aliased signal power spectrum that differs from ${a}_{\parallel}$.
- 4. The control law transfer function for the Keck AO control system, based on a third-order proportional, derivative and integration (PID) algorithm, is given in the discrete domain by (see van Dam
*et al.*[16]):$$C(z)=\frac{{a}_{0}+{a}_{1}{z}^{-1}+{a}_{2}{z}^{-2}+{a}_{3}{z}^{-3}}{1+{b}_{1}{z}^{-1}+{b}_{2}{z}^{-2}+{b}_{3}{z}^{-3}},$$where the Z-transform argument $z$ is replaced by $\mathrm{exp}\text{\hspace{0.17em}}i2\pi \nu /{\nu}_{s}$ when we want to express this law in the continuous domain. The coefficients ${a}_{i}$ and ${b}_{i}$ are set by design. ${a}_{0}$ is the proportional loop gain, and ${b}_{1}$ is set to a value slightly smaller than 1 (leaky integrator gain) in order to avoid invisible modes building up in the DM commands. Note that the control law might vary between systems. Our objective here is only to demonstrate the noise-reduction technique on a particular case, the Keck AO system. - 5. The DM transfer function is assumed to be 1, as the first mechanical resonance frequency is much higher than the usual bandwidth of the AO system.
- 6. The zero-order hold transfer function is actually 1 because it is already included in the WFS integration time.

From inspection of the loop diagram in Fig. 11, we find, for the amplitude spectrum of the command of actuator $k$ (we skip the frequency in the notation to improve the reading and ∼ indicates a Fourier transform),

- • $w{n}_{k}$ is the $k$th component of the transformation of the slope white noise vector ${\mathbf{S}}_{n}$ into a white noise actuator command vector (by application of the command matrix ${D}^{+}$), and is related to the noise command ${n}_{k}$ we are interested in Eq. (12) by
- • ${\tilde{a}}_{\parallel ,k}$ is the spectrum of the turbulent phase projected onto actuator $k$.

From Eq. (17), we can now compute the cross-spectrum of the DM commands ($\star $ indicates complex conjugation),

- 1. From the telemetry, compute the actuator command temporal power cross-spectrum $\u27e8{\tilde{m}}_{k}{\tilde{m}}_{l}^{\star}\u27e9$;
- 2. Build the closed loop and WFS transfer functions;
- 3. Multiply the commands cross-spectrum with the ratio ${|W/{H}_{\mathrm{CL}}|}^{2}$ to make the high-frequency ${\tilde{wn}}_{k}{\tilde{wn}}_{l}^{\star}$ white noise cross-spectrum plateau evident;
- 4. At this point, we are left with a temporal cross-spectrum term equal to$$\u27e8{\tilde{a}}_{\parallel ,k}{\tilde{a}}_{\parallel ,l}^{\star}\u27e9{|W|}^{2}+\u27e8{\tilde{wn}}_{k}{\tilde{wn}}_{l}^{\star}\u27e9.$$Here, there are two options: either we have some clue about the atmospheric signal, in which case we can adjust a model to the spectrums ${\tilde{a}}_{\parallel ,i}$; or we do not, in which case (a) we use the fact that the atmospheric signal is low at high frequency and is dominated by noise, which is generally true unless we work at very low sampling frequency, and (b) we evaluate the average level of the high-frequency plateau (Fig. 12). This is actually the easiest way. Indeed, knowing the atmospheric signal would actually require us to know not only the seeing, but also the vertical distribution of the turbulence and wind velocity. In this case, we would not need the DM-based seeing monitor.
Therefore, we have no choice but to assume that the noise dominates the signal at high frequency, and make a reasonable choice of the frequency above which the average of the t-PSD plateau is computed. Of course, it is possible to iterate: get a first estimate of the seeing, assume a certain vertical distribution of the OT and wind profiles, compute the terms ${\tilde{a}}_{\parallel ,k}$, and redo the estimation. Due to the large number of assumptions that are needed in such a case, it remains to be demonstrated that this iterative procedure brings any advantage.

- 5. Once the white noise t-PSD plateau level $\u27e8{\tilde{wn}}_{k}{\tilde{wn}}_{l}^{\star}\u27e9$ is estimated, the command noise covariance can be computed using Parseval equality:$$\u27e8{n}_{k}{n}_{l}\u27e9\equiv \frac{1}{T}{\int}_{t}{n}_{k}(t){n}_{l}(t)\mathrm{d}t=\frac{1}{T}{\int}_{\nu}{\tilde{n}}_{k}(\nu ){\tilde{n}}_{l}^{\star}(\nu )\mathrm{d}\nu \phantom{\rule{0ex}{0ex}}=\u27e8{\tilde{wn}}_{k}{\tilde{wn}}_{l}^{\star}\u27e9{\int}_{\nu}{\left|\frac{{H}_{\mathrm{CL}}(\nu )}{W(\nu )}\right|}^{2}\mathrm{d}\nu .$$This procedure is illustrated in Fig. 12 on the power spectrum ($k=l$) of a noisy signal. The white noise plateau is well apparent. We also compare in Fig. 13 the estimated actuator noise as a function of the number of photoelectrons detected during the WFS integration over the full detector, with the theoretical law in $1/\sqrt{{N}_{\gamma}}$. This is done for about 100 data sets, over three nights, with a ratio of high to low photon flux of 100. The transfer function was adapted for the-loop frame for each case, and the range was 140–1054 Hz. The agreement is remarkable, even for a low light level.

#### B. Compensating the Contribution of the Wavefront Sensor Spatial Aliasing

The three last terms on the right member of Eq. (13) show the contribution of the WFS spatial aliasing that we need to remove from the DM command covariance,

*et al.*[6], where they propose to run a first ${w}_{0}$ and ${L}_{0}$ estimate ignoring the aliasing term, then use a model of the aliasing term adjusted for the seeing amplitude and do a second estimation taking into account the aliasing term. According to Kolb

*et al.*this technique quickly converges and produces good results.

We think it is not necessary to apply an iterative procedure. Indeed, the effect of aliasing is to systematically increase the DM command variance in relation to the number of lenslets or degree-of-freedom of the WFS, independently of the seeing amplitude. Thus, because this effect is systematic, we can compute numerically, using a Monte Carlo procedure, the variance of the WFS aliasing expressed in the modified Zernike coefficients for $D/{r}_{0}=1$, and simply compensate the Zernike variances for this factor.

Figure 14, top, shows the RMS of the aliasing Zernike coefficients and the RMS of the OT Zernike coefficients for a seeing of ${0.7}^{\prime \prime}$, 20 m outer scale, and the Keck AO system DM pitch. The computation is based on the generation of random phase screens drawn from the aliasing spatial power spectrum given in Jolissaint [17]. These phase screens are then projected in the modified Zernike basis. The top figure shows that the aliasing RMS is quite constant over a large range of indexes until it drops. Therefore, as the OT Zernike variance decreases steeply with $j$, the relative effect of aliasing increases towards the high orders until a maximum is reached (bottom of Fig. 14). We have computed the latter curve for different values of the outer scale (down to 10 m) and saw essentially no change.

Therefore, it is possible to build an empirical model of the variance excess as a function of the radial order, and compensate this excess on the modified Zernike variance after the noise has been compensated. Our model is an order 4 polynomial, fit to the curve shown in Fig. 14, valid for the range $n=3$ to $n=15$:

with## 7. ON-SKY VALIDATION

The PSF-R project at Keck Observatory started in 2007 with Flicker [18], and was continued and completed by our team from 2009 to 2017. During this period, more than 1000 AO runs were dedicated to the project and 841 were considered valid to test our PSF-R algorithm. Of these, 636 runs are in NGS mode and 205 in LGS mode. There is no conceptual difference between the two modes as far as seeing estimation is concerned. In reality, a close inspection of the modified Zernike coefficient variances showed some unexpected behavior in the LGS case, requiring further rejection of some modes. At the end, the seeing estimation quality is no different for the two modes.

In this section, after a summary of the OT estimation procedure, we will show how well the variance model we developed fits the data, and then show a comparison of the estimated seeing with a simultaneous seeing estimated from on-loop PSF on the same telescope, as well as with differential image motion monitor (DIMM) and multi-aperture scintillation sensor (MASS) seeing measurements.

#### A. Calculating the Seeing Angle and Outer Scale, in Practice

The procedure to get the seeing angle and the outer scale from the raw DM commands is summarized as follows:

- 1. Identify and discard the bad modes by inspection of the modified Zernike variances for a significant number of AO experiments. This point is very critical and requires some analysis of the system’s behavior;
- 2. Remove the average from the DM commands in case an offset is applied on the DM to compensate for nonturbulent static aberrations. Optical turbulence wavefront error shall be null on average;
- 3. Compute the DM command covariance $\u27e8{\mathbf{mm}}^{t}\u27e9$;
- 4. Estimate the DM command noise covariance $\u27e8{\mathbf{nn}}^{t}\u27e9$ and subtract it from the DM command covariance;
- 5. Transform the noise-free DM command covariance into the modified Zernike coefficient covariance matrix;
- 6. Remove the bad modes from the modified Zernike covariance matrix;
- 7. Keep only the
*variances*of the modified Zernike coefficients; indeed it makes enough valid data points, and in any case the outer-scale damping model is only for the variances; - 8. Apply the aliasing correction factor to the modified Zernike variances;
- 9. Construct the normalized variance model, i.e., for $D/{r}_{0}=1$ and an infinite outer scale;
- 10. Divide the modified Zernike variances by the normalized model for each mode individually;
- 11. Find the ratio $D/{L}_{0}$, making the preceding ratio independent of the j-index, using the empirical model for the outer-scale damping ${\mathcal{W}}_{j,j}$; this provides the best estimation of the outer scale;
- 12. Divide once again the modified Zernike variances by the normalized model for each mode individually, but now including the outer-scale damping; we get, for each good mode, a value of the ${(D/{r}_{0})}^{5/3}$ ratio from which we extract a series of ${r}_{0}$ values, and then the average ${r}_{0}$ and ${w}_{0}$ for the AO run, as well as an estimate of the uncertainty on ${r}_{0}$, given by
- 13. As a verification, find the ${(D/{r}_{0})}^{5/3}$ ratio that minimizes the total quadratic difference between the variances and the model, including the effect of the outer scale; in principle, this best fit ${(D/{r}_{0})}^{5/3}$ should be compatible with the average value computed from individual modes, i.e., it must be within the $1\text{-}\sigma $ dispersion range; if not, it is a sign that there might be an issue with the data.

#### B. Comparison of the Variance Model and the Measured Variances in NGS Mode

The estimation of ${w}_{0}$ and ${L}_{0}$ are made via a fit of the modified Zernike variance model to the DM command variances. Therefore, it is critical to verify how well the model represents the real data. In order to do this, we normalized the variances by the factor ${(D/{r}_{0})}^{5/3}{\mathcal{W}}_{j,j}(D/{L}_{0})$, using for each set the Fried parameter and outer scale determined from the set itself. Figure 15 shows the comparison of the model variances, in green and black, and the average variances for the 636 NGS cases. From this, we concluded the following:

- • The model is consistent with the data on average (see in particular the zoomed graph). The $1\text{-}\sigma $ error bars almost always include the model prediction except for the modes for which the azimuthal index $m=1$, where the model is strangely almost two orders of magnitude above the data. We were not able to identify the origin of the discrepancy, and we simply considered all $m=1$ modes as bad modes, excluded from the fit. The purely radial modes, $m=0$, while compatible with the data, showed also a systematic trend; because we had so many valid modes, we decided to also discard these modes in order not to add a possible systematic error.
- • Beyond the j-index 120, we found that the difference between the model and the average variance was getting gradually larger than the $\sigma $ of the variances (not for all the modes, though), so we decided to reject the modes $j>120$ as well. Overall, we were left with 90 valid modes in NGS mode, in the range $j=9$ to $j=120$.
- • For a significant fraction of the modes, Noll’s model is perfectly suitable, i.e., the representation and sampling errors do not affect the quality of the model. Once these modes are identified, it should be safe to use the simpler Noll model, with Winker factor, for the fit.

Figure 16 shows an example of model fitting to the data after final selection of the good modes. There is some statistical noise, certainly due to the fact that the AO run exposures were limited in time: indeed, only a stationary optical turbulence and an infinite exposure time would make the measurements perfectly identical to the OT models we are using. Overall, though, the model matches the data because the residual statistical error due to the limited exposure time has a random sign for each mode, and gets somewhat cancelled in the fitting process.

It is important to note here that because the seeing changes relatively rapidly, the seeing we are estimating can only be an average of the seeing during the AO run science exposure. If the seeing has changed too much, i.e., if the turbulence has been too unstable during the AO run, there will be an excess variance in the DM data with respect to the model we use. This is clearly a limit of the current method, which assumes spatial and temporal stationarity of OT. We have been made aware that techniques to extract the OT parameters for a non-stationary turbulence have been developed (see Brennan and Mann [19] and references therein). Implementation of these techniques in the context of astronomical AO application is left to another study.

#### C. Comparison of the Variance Model and the Measured Variances in LGS Mode

Figure 17 shows the comparison of the variance model and the average variance in LGS mode for the 205 sets. Overall, the match between the model and the data is good, and the same comments as for the NGS mode apply except for one thing: for all the Zernike polynomials whose azimuthal component is in $\mathrm{cos}\text{\hspace{0.17em}}m\theta $, with $m$ even, there is a systematic excess of variance as shown in Fig. 17, bottom. This behavior is not seen at all in NGS mode. Thus, there is some systematic error in LGS mode whose origin we have not been able to identify. We do not think it is coming from the data processing, as we process the NGS and LGS the same way. It should therefore come from the LGS data itself. Keck AO is a zonal system, so we cannot expect the actuator control itself to be at the origin of this asymmetry.

The objective of the seeing estimation project is not to debug the AO system, however, so therefore we simply decided to discard the Zernike modes impacted by this effect. We were therefore left with 75 good LGS modes, still enough for an accurate determination of OT. Note that before suppressing these modes, the estimated seeing in LGS mode was systematically about 15% higher than in NGS mode. This example shows the importance of a close inspection of the system behavior in order to avoid spurious systematic errors in the seeing determination.

#### D. WFS Noise and Error in the Seeing Determination

If the noise contribution to the variance has been well subtracted, then there should be no correlation between the WFS noise level and the error in the seeing angle. Figure 18 shows that such is indeed the case for both NGS and LGS, for all 841 data sets. The noise variance in LGS mode is simply higher than in NGS mode, but there is no correlation with the relative error in the seeing angle.

#### E. Seeing Angle and Outer-Scale Statistics

We show in Figs. 19 and 20 the seeing angle and outer-scale distribution for all the 841 valid measurements done during the PSF-R project. The mean and median seeing are in line with the values measured for this site, albeit slightly larger. This might be due to some mirror seeing on the telescope. According to Zago [20], what is called dome seeing is actually essentially coming from a temperature difference between the primary mirror and the immediate layer or air above it. The rule of thumb, validated on many telescopes, is that a difference of $+1\xb0\mathrm{C}$ between the air and the mirror generates ${0.1}^{\prime \prime}$ of seeing. We also see that there is no statistical difference between the results obtained in NGS and LGS mode. Note that the uncertainty on the seeing ($1\sigma $) for each measurement was ${0.03}^{\prime \prime}$ on average.

The outer scale we are measuring is on average a factor 2 smaller than
values measured on the same summit on a nearby 8 meter
telescope (Subaru) by the RAVEN AO system team (Ono
*et al.* [21]) on different nights (which might also explain some part
of the observed discrepancy). Using a SLODAR technique, they were able
to retrieve vertical profiles of the outer scale and show that this
parameter varies significantly with the altitude in a range
15–35 m, on average. The SLODAR technique, as it makes
use of the correlation of the slopes from an array of
Shack–Hartmann lenslets, is in principle relatively insensitive
to the limited size of the telescope aperture while, in the DM-seeing
method, we determine the outer scale from the effect it has on the
Zernike modes variances over the full aperture. Therefore, our
outer-scale determination method critically depends on the pupil
diameter, and it might be that the SLODAR and DM-seeing methods do not
measure the outer scale with the same biases. This assumption remains
to be explored.

We tried to *force* the outer scale to 20 m in
the data reduction, but found that the fitting of the model with the
low-order variances was not satisfactory, with a model variance always
larger than the measurements at low order. As a consequence, the
estimated seeing angle was lowered by about 5%.

In our opinion, here we are reaching the limits of the optical
turbulence theory. The assumption that the turbulent flow is
homogeneous and the scales of the eddies distributed according to the
Kolmogorov law (see Tatarski for details) is truly valid in the free
atmosphere, but ground structures like the observatory building and
dome certainly break the homogeneity of the flow, where optical
turbulence is at its strongest. Also, if the turbulence is at its
strongest—near the ground and the dome slit—where the
outer scale is at its lowest, it should not be a surprise that the
outer scale measured on the *full aperture* tends to be
equal to the slit and mirror diameter (see Eq. 12 in Ono
*et al.*, showing how the optical outer scale is
weighted by the optical turbulence profile). This is of course only an
assumption needing further exploration as well.

Besides, we must not forget, as Voitsekhovich and Cuevas [22] remind us, that the von Karman
model of the turbulent phase spatial power spectrum, at the basis of
all models used so far to account for the damping effect of the outer
scale, is *only* a simple *empirical
model* not based on physics. In fact, other models exist that
produce, according to Voitsekhovich, similar results as the VK
model.

What we can finally conclude from our measurements is that there is a significant damping of the low-order aberration variances with respect to a fully developed Kolmogorov flow, and that this damping is relatively well described by the von Karman empirical modification of the Kolmogorov model using an outer scale of 13 m on average. Deciding if the VK model is appropriate or not to describe the real shape of the turbulent flow near the telescope aperture, or if a more sophisticated inhomogeneous model shall be used, is beyond the scope of this work.

#### F. Note on the Number of Good Modes and the Seeing Determination Accuracy

The selection of the first good mode has an impact on the accuracy with which the outer scale is estimated: the lower the j-index, the stronger the ${L}_{0}$-damping effect and therefore the better the accuracy on ${L}_{0}$, which has an immediate impact on the seeing determination (ignoring the problem with the potential instrument aberrations); on the other hand, the higher the last j-index, the more data points we have and the better the seeing estimation, but we also need to remove bad modes here and there for various reasons. As we have seen, we are left with 90 data points in NGS mode and 75 in LGS mode. In principle, as we have de-trended the measured variance from the outer-scale damping, when we divide the variances with Noll’s model, we should see a straight line. The average of this line gives the mean ${(D/r0)}^{5/3}$, from which we get ${r}_{0}$ and the seeing. As the departure from Noll’s model has many different (unknown) origins, we expect the error distribution to be Gaussian, so the RMS on the mean ${r}_{0}$ shall be given by the classical formula $\sigma (\text{mean})=\text{mean}(\sigma (\text{data}))/\sqrt{\mathrm{nb}.\text{data}}$. So, going from 90 to 75 data points (NGS to LGS), the relative loss is 10%, i.e., the statistical error of the seeing determination from the LGS data is only 10% larger than that for the NGS case.

But all this is relatively theoretical. Indeed, in our understanding, the accuracy of the OT parameters has more to do with the selection of the good modes than with the number of modes, assuming of course that there are enough valid modes. As we see in the Keck example, a few tens of modes well distributed from the very first to last mode shall be enough. For any other system, we would therefore recommend carefully examining the behavior of the modal variances to select the appropriate modes. Also, the only solid way to estimate the accuracy of the seeing determination is to run a AO on/off experiment as we describe next.

#### G. Comparison with Open-Loop PSF Seeing

Certainly the most important test of our method was the direct comparison of the DM-monitor seeing with the seeing extracted from simultaneous on-sky open-loop PSF, acquired on the same telescope, on the same optical axis. In such a case, it is expected that both seeings should be highly correlated. During two nights, June 22 and 23 of 2011, we did 96 AO runs alternated with 75 open-loop PSF acquisitions. The typical delay between a closed-loop and an open-loop run was 90 s. This delay was too long to catch the same turbulent conditions, but short enough to catch the long term evolution of the atmosphere over a few minutes, and verify that the seeing evolution were identical in both time series (Fig. 21).

On the Keck AO system, the open-loop mode is actually a tip-tilt correction mode, so the seeing cannot be estimated directly from the long exposure PSF themselves. Furthermore, there is the PSF improvement due to the outer scale. We therefore built, using a Monte Carlo simulation of tip-tilt correction, a library of synthetic long exposure tip-tilt corrected PSF, for a range of seeing values from ${0.2}^{\prime \prime}$ to ${1.9}^{\prime \prime}$, and an outer scale equal to the average we have measured in our data set, 12 m. Then, after careful cosmetic cleaning of the sky open-loop PSF (acquired with the near infrared camera NIRC2, at 1.65 μm) we looked into the library for the PSF whose Strehl ratio was the closest to the sky PSF Strehl. We knew that there were some uncorrected static aberrations in the system (about 150 nm), but their contribution on the tip-tilt corrected PSF Strehl was negligible.

The median rate of change of the seeing during the experiments was, measured from the seeing time series themselves, 0.3 mas/s, in both directions (increase or decrease of the seeing). Therefore, the seeing change from a closed-loop to an open-loop run, separated by 90 s, was on average $\pm {0.03}^{\prime \prime}$. Adding to this the uncertainty of ${0.03}^{\prime \prime}$ of the seeing angle, we end up with a probable deviation of $\pm {0.06}^{\prime \prime}$ between a DM-based seeing measurement and the subsequent PSF-based seeing measurement. In Figs. 21, we show the superposed time evolution of the DM- and PSF-based seeing measurements for the two on-sky experiments of June 22 and 23, 2011. Most of the PSF seeing is within the expected range, so we can conclude that the two series are compatible, which is also apparent in their statistics, given in Table 2.

We show the correlation between the DM and PSF seeing measurements in Fig. 22, where the DM-based seeing measurements has been interpolated at the same time as the PSF measurements. The correlation coefficient is 90.6% and the slope between the two series is 1.02. Most of the DM-based seeing values are within $\pm 10\%$ of the PSF-based values. In our opinion, these results fully validate the DM-seeing monitor technique.

#### H. Comparison with DIMM and MASS Seeing

The last test we did was comparing the DM-based seeing with
simultaneous measurements made by the Mauna Kea seeing
monitors’ *differential image motion monitor*
(DIMM, Sarazin and Roddier [23]) and *multi aperture scintillation sensor*
(MASS, Kornilov *et al.* [24]). The DIMM and MASS data are taken from the Mauna
Kea Weather Center web site archive (http://mkwc.ifa.hawaii.edu/current/seeing/).
Figure 23 shows the
night average seeing for the DM/DIMM/MASS monitors for nine nights
between June 2011 and July 2015.

It appears that for most of the nights, the DM and DIMM night average
seeing are similar. This is not the case for the MASS seeing, which is
for five over eight nights significantly smaller than the DM values,
but this is expected, as it is known that the MASS monitor is not
sensitive to the low-altitude layers. These measurements can be
interpreted as an indication that most of the time there is a
significant ground layer of optical turbulence at Mauna Kea, a fact
that is well known (see for instance Ono
*et al.* [21] and reference herein).

That being said, while the correlation between the nightly average can be relatively good, at least with the DIMM values, this is not the case on a shorter time basis (minutes/hours). This is demonstrated by the RMS of the difference between the DM, DIMM, and MASS seeing shown in Table 4: the DIMM-DM dispersion is four times larger than the open-loop PSF-DM dispersion, and the MASS-DM one is six times larger.

Also, there can be important differences between the DM, DIMM, and MASS
seeing from one night to another as well as during a given night. The
night of June 22, 2011 (Fig. 24) shows a case in which the average seeing was relatively
similar, but we can see that this correlation was *mostly
coincidental*. The night of September 14, 2013
(Fig. 25) shows an
example where the three monitors were producing totally different
results.

Figure 26 shows the DIMM, MASS, and OL PSF seeing versus the DM seeing. It is obvious that the OL PSF seeing is much more correlated with the DM seeing than are the DIMM and MASS seeing. This is a clear demonstration of the fact that the seeing changes significantly from a place to another because of the influence of the ground structures on the turbulent flow, which might change with the wind direction, and because the monitors are not sensing the turbulence from the same altitude. Besides, mirror seeing can be present.

As a consequence, we think that an accurate estimation of the seeing
associated with a given AO observation requires a measurement
collocated with the system optical axis at the same time as the
observation. The DM-seeing monitor is the ideal solution for this. A
distant seeing monitor cannot produce any useful results in that
respect, and shall only be used for site monitoring. *It might
even be that, because of the poor correlation between seeing monitors
and the telescope seeing measured by the DM seeing, observation
scheduling based on seeing prediction would only have a limited
interest* unless the very local and mirror seeing could be
predicted. It is therefore essential, in our understanding, to
collocate as much as possible the OT monitors and the optical axis of
the telescope.

## 8. CONCLUSIONS

We have presented a detailed analysis of the systematic and random errors affecting the determination of the seeing angle and outer scale of optical turbulence from the deformable mirror commands of an adaptive optics system. Mitigation or correction solutions have been proposed for each term. The method was developed for both the natural and laser guide star modes of the adaptive optics system of the W. M. Keck-II telescope, and can be applied to any other system. The method has been validated with a comparison to simultaneous measurements of the seeing from open-loop PSF acquired on the same optical axis. We have demonstrated that an accuracy of 10% on the determination of the local and instantaneous seeing is achievable.

We tested the DM wavefront for stationarity in the pupil, a particularly important requirement in this method, and showed that the pupil edges had to be avoided by selecting an annulus inside the telescope pupil for the seeing determination. Zernike projection error on the DM influence function basis and numerical representation error have been identified and compensated by a proper use of a modified Zernike polynomial basis. A noise covariance determination from the power spectrum of the actuator commands has been developed, and comparison of the noise amplitude with the expected behavior as a function of the number of photons per integration time has validated the approach. A simple method to compensate for wavefront sensor aliasing has been developed.

The statistical analysis of the DM commands expressed in the modified Zernike basis demonstrates the validity of the Kolmogorov model, with the von Karman empirical modification, to describe the Keck AO system behavior. This in turn indicates that, by using this system, it is possible to extract the OT parameters from the telemetry (which is not the case on all systems) because non-Kolmogorov sources of dynamical errors are negligible. That being said, in LGS mode, an unexpected excess of dynamical aberration was identified for modes with an even azimuthal order $m$. These modes were simply discarded from the seeing determination process. After this correction, no statistical difference between the natural and laser guide star modes was found.

The statistics of the seeing angle determined for 841 AO runs from 2011 to 2015 is similar to the expected value for the Mauna Kea summit, about ${0.6}^{\prime \prime}$. The estimated outer scale is smaller than the average value measured with other monitors. We postulate without proof that this might be due to the difference between the geometry of the outer-scale determination in the DM-seeing approach, where the whole pupil is used, and the other methods, which study the statistics of the wavefront as a function of the separation vector within the pupil. The dome structure might also have an influence.

We finally demonstrated that the DM-based seeing was highly correlated with the seeing measured on the same axis, at a quasi-simultaneous time, on open-loop PSF. The relative difference was 10% on average, with a dispersion of only ${0.06}^{\prime \prime}$. This method is therefore valid for retrieving accurate seeing determination values, a particularly critical aspect for system performance validation and other sensitive post-AO methods such as PSF reconstruction. On the contrary, the difference between the DM and DIMM seeing can be more than 40%, and more than 80% with the MASS seeing. It is therefore highly recommended not to use non-collocated seeing monitors when accurate seeing estimation is required at a given telescope.

## Funding

National Science Foundation (NSF) (AST-1207631); Association of Canadian Universities for Research in Astronomy (AURA); California Association for Research in Astronomy (CARA); Rijksuniversiteit Groningen, the Netherlands.

## Acknowledgment

The authors acknowledge the help of Chris Neyman with data acquisition at the W. M. Keck telescope, as well as the many fruitful comments made by the anonymous reviewers of the current paper and of an earlier version of this paper. We also would like to thank Daniel Winker for providing us the outer-scale attenuation data from his analytical model. Data presented herein were obtained at the W. M. Keck Observatory, which is operated as a scientific partnership between the California Institute of Technology, the University of California, and the National Aeronautics and Space Administration. The observatory was made possible by the generous financial support of the W. M. Keck Foundation. The authors recognize and acknowledge the significant cultural role and reverence that the summit of Mauna Kea has always had within the indigenous Hawaiian community. We are most fortunate to have had the opportunity to conduct observations from this mountain.

## REFERENCES

**1. **F. Rigaut, G. Rousset, P. Kern, J.-C. Fontanella, J.-P. Gaffard, F. Merkle, and P. Léna, “Adaptive optics on a
3.6-m telescope—results and
performance,” Astron.
Astrophys. **250**,
280–290
(1991).

**2. **J.-P. Véran, F. Rigaut, H. Maître, and D. Rouan, “Estimation of the
adaptive optics long-exposure point spread function using control loop
data,” J. Opt. Soc. Am. A **14**, 3057–3069
(1997). [CrossRef]

**3. **J. R. Lu, A. M. Ghez, S. Yelda, T. Do, W. Clarkson, N. McCrady, and M. Morris, “Recent results and
perspectives for precision astrometry and photometry with adaptive
optics,” Proc. SPIE **7736**, 77361I
(2010). [CrossRef]

**4. **M. Schöck, D. Le Mignant, G. A. Chanan, P. L. Wizinowich, and M. A. van Dam, “Atmospheric turbulence
characterization with the Keck adaptive optics systems. I. Open-loop
data,” Appl. Opt. **42**, 3705–3720
(2003). [CrossRef]

**5. **T. Fusco, G. Rousset, D. Rabaud, E. Gendron, D. Mouillet, F. Lacombe, G. Zins, P.-Y. Madec, A.-M. Lagrange, J. Charton, D. Rouan, N. Hubin, and N. Ageorges, “NAOS on-line
characterization of turbulence parameters and adaptive optics
performance,” J. Opt. A **6**, 585–596
(2004). [CrossRef]

**6. **J. Kolb, N. Muller, E. Aller-Carpentier, P. Andrade, and J. Girard, “What can be retrieved
from adaptive optics real-time data?”
Proc. SPIE **8447**,
84475U (2012). [CrossRef]

**7. **V. Tatarski, *Wave Propagation in a Turbulent
Medium* (Dover,
1961), translated by R. A. Silverman.

**8. **F. Roddier, “The effect of
atmospheric turbulence in optical astronomy,”
in *Progress in Optics*, E. Wolf, ed.
(North-Holland, 1981),
Vol. XIX,
pp. 281–376.

**9. **D. L. Fried, “Optical resolution
through a randomly inhomogeneous medium for very long and very short
exposures,” J. Opt. Soc. Am. **56**, 1372–1379
(1966). [CrossRef]

**10. **C. E. Coulman, “Fundamental and applied
aspects of astronomical seeing,” Appl.
Opt. **23**,
19–57 (1985). [CrossRef]

**11. **R. Conan, R. Avila, L. J. Sánchez, A. Ziad, F. Martin, J. Borgnino, O. Harris, S. I. González, R. Michel, and D. Hiriart, “Wavefront outer scale
and seeing measurements at San Pedro Mártir
observatory,” Astron.
Astrophys. **396**,
723–730
(2002). [CrossRef]

**12. **A. Ziad, J. Maire, J. Borgnino, W. Dali Ali, A. Berdja, K. Ben Abdallah, F. Martin, and M. Sarazin, “MOSP: monitor of outer
scale profile,” in *Adaptative Optics
for Extremely Large Telescopes (AO4ELT-I)*,
2010.

**13. **R. J. Noll, “Zernike polynomials and
atmospheric turbulence,” J. Opt. Soc.
Am. **66**,
207–211
(1976). [CrossRef]

**14. **D. M. Winker, “Effect of a finite
outer scale on the Zernike decomposition of atmospheric optical
turbulence,” J. Opt. Soc. Am.
A **8**,
1568–1573
(1991). [CrossRef]

**15. **L. Jolissaint, J.-P. Veran, and J. Marino, “OPERA, an automatic PSF
reconstruction software for Shack-Hartmann AO systems: application to
Altair,” Proc. SPIE **5490**, 151–163
(2004). [CrossRef]

**16. **M. A. van Dam, A. H. Bouchez, D. Le Mignant, E. M. Johansson, P. L. Wizinowich, R. D. Campbell, J. C. Y. Chin, S. K. Hartman, R. E. Lafon, P. J. Stomski Jr., and D. M. Summers, “The W. M. Keck
observatory laser guide star adaptive optics system: performance
characterization,” Publ. Astron. Soc.
Pac. **118**,
310–318
(2006). [CrossRef]

**17. **L. Jolissaint, “Synthetic modeling of
astronomical closed loop adaptive optics,”
J. Eur. Opt. Soc. Rapid Publ. **5**,
10055 (2010). [CrossRef]

**18. **R. Flicker, 2008, https://www.lao.ucolick.org/static/PsfReconstruction/psf_rev.pdf.

**19. **T.-J. Brennan and D. C. Mann, “Estimation of optical
turbulence characteristics from Shack Hartmann wavefront sensor
measurements,” Proc. SPIE **7816**, 781602
(2010). [CrossRef]

**20. **L. Zago, “Engineering handbook
for local and dome seeing,” Proc.
SPIE **2871**,
726–736
(1997). [CrossRef]

**21. **Y. H. Ono, C. M. Correia, D. R. Andersen, O. Lardière, S. Oya, M. Akiyama, K. Jackson, and C. Bradley, “Statistics of
turbulence parameters at Mauna Kea using the multiple wavefront sensor
data of RAVEN,” Mon. Not. R. Astron.
Soc. **465**,
4931–4941
(2017). [CrossRef]

**22. **V. V. Voitsekhovich and S. Cuevas, “Adaptive optics and the
outer scale of turbulence,” J. Opt.
Soc. Am. A **12**,
2523–2531
(1995). [CrossRef]

**23. **M. Sarazin and F. Roddier, “The ESO differential
image motion monitor,” Astron.
Astrophys. **227**,
294–300
(1990).

**24. **V. Kornilov, A. Tokovinin, N. Shatsky, O. Voziakova, S. Potanin, and B. Safonov, “Combined MASS-DIMM
instruments for atmospheric turbulence
studies,” Mon. Not. R. Astron.
Soc. **382**,
1268–1278
(2007). [CrossRef]