## Abstract

Several phase-modulation functions have been reported to decrease the aberration variance of the modulation-transfer-function (MTF) in aberration-tolerant hybrid imaging systems. The choice of this phase-modulation function is crucial for optimization of the overall system performance. To prevent a significant loss in signal-to-noise ratio, it is common to enforce restorability constraints on the MTF, requiring trade of aberration-tolerance and noise-gain. Instead of optimizing specific MTF characteristics, we directly minimize the expected imaging-error of the joint design. This method is used to compare commonly used phase-modulation functions: the antisymmetric generalized cubic polynomial and fourth-degree rotational symmetric phase-modulation. The analysis shows how optimal imaging performance is obtained using moderate phase-modulation, and more importantly, the relative merits of the above functions.

© 2010 OSA

## 1. Introduction

The use of pupil-plane phase-modulation in hybrid optical-digital imaging systems can enable good imaging quality even in the presence of high levels of optical aberration, such as defocus, astigmatism and spherical aberration [1–4]. We use the term hybrid imaging to refer to a system that employs a modification of the point-spread function (PSF) to optically encode an image together with digital decoding. This process of optical encoding and digital decoding is analogous to the use of a codec to enhance transmission of a signal through an imperfect telecommunication channel: in the case of hybrid imaging, the aim is to reduce sensitivity to aberrations in the optics that would otherwise reduce image quality. In earlier work a hybrid imaging system was designed for improved object identification [5]. In this paper we introduce a method to find the phase-profile and modulation depth that maximize the expected image fidelity for a required aberration tolerance. Crucial for such a hybrid design is a well-considered choice of phase-profile used for optical encoding. We describe here the optimization of the most important phase-modulations for reduced sensitivity to defocus; namely third-order antisymmetric profiles and radially-symmetric quartic phase-modulation. However the technique is more generally applicable to higher order aberrations, as well as to other phase- or amplitude-modulations.

Various types of phase-modulations have been shown to increase tolerance to defocus; one of the most common is the cubic phase-modulation and its generalizations of the form:

*x*and

*y*are normalized pupil coordinates, and the parameters

*α*and

*β*characterize the phase-modulation. Phase-masks such as the separable logarithmic [6], the exponential [7] and the generic polynomial phase-mask [8] have profiles similar to the pure-cubic phase-profile; that is, with

*β*≡0. Also non-separable profiles, such as those described by Eq. (1) with

*β*≈-3

*α*have been found to yield systems with good defocus tolerance [9,10].

Also of interest are radially-symmetric phase profiles such as the logarithmic asphere [11], the quartic phase-mask [3,4] and spherical coding [12]. The latter two can be described as:

*ρ*is the normalized radius, and the parameters

*γ*and

*δ*characterize the phase-modulation.

In the next section we describe how maximum-likelihood image restoration can be used to estimate the expected imaging-error due to aberration, noise, and image restoration in hybrid imaging systems. In section 3 we evaluate and compare imaging performance for phase-profiles described by Eq. (1) and Eq. (2). For the former class of phase profiles, two regimes of third-order phase-profiles are found to yield optimal imaging performance, and we show that for a given focal-depth extension, an optimal balance exists between defocus-tolerance and the reduction in signal-to-noise ratio due to suppression of the modulation-transfer function (MTF) by the pupil-plane phase-modulation. The imaging performance of these third-order anti-symmetric optima is compared to that of the radially-symmetric quartic phase-modulation. Finally, in section 4 the validity of the model is tested on a set of scenes using a structural-similarity image metric that has shown good agreement with the perceived similarity of images in psychophysical experiments [13].

## 2. Hybrid imaging fidelity evaluation

A hybrid imaging system can be considered to perform well if the final restored image is similar to the ideal noise-free, diffraction-limited image. To quantify performance, the expected mean-squared error between the ideal and restored images is calculated over the required aberration tolerance range. We define the imaging fidelity here as the inverse of the expected mean-squared error. This mathematically convenient definition allows the use of a general scene- and noise-model, enabling efficient numerical optimization.

One can write the expected mean-squared error, ${\overline{\epsilon}}^{2}$of a shift-invariant system as a function of the Fourier-transform of the restored image, *I _{r}*, and the noiseless diffraction-limited image,

*I*:

_{dl}*H*is the aberrated optical-transfer-function(OTF) including pupil phase-modulation,

_{ab}*H*is the Fourier transform of the image restoration filter (for example a Wiener filter), and

_{W}*H*is the diffraction-limited OTF. The Fourier-transforms of scene and noise are represented by

_{dl}*S*and

*N*respectively, $\ufffd(\cdot )$ denotes the expectation value, ${\u3008\cdot \u3009}_{{f}_{X},{f}_{Y}}$and ${\u3008\cdot \u3009}_{ab}$indicate the ensemble averages over all spatial frequencies and the aberration tolerance range respectively. Equation (3) can be simplified for signal- and noise-spectra modeled as independent Gaussian signals with zero mean as:

*H*is a function of

_{W}*H*and

_{ab}*P*[14], knowledge of the variance of the scene spectrum,

_{S}/P_{N}*P*, and noise spectrum,

_{S}*P*is sufficient to predict the imaging error of a given system. Often the spatial frequency spectrum of the noise is modeled as white and Gaussian, in which case the comparison of hybrid imaging systems can be based solely on

_{N}*H*and

_{ab}*P*. While the latter is scene-dependent, a representative measure of

_{S}/P_{N}*ε*can be obtained by incorporation of the typical amplitude spectrum of scenes: this is approximately proportional to

*1/|f|*, where

^{κ}*|f|*is the spatial frequency normalized to the optical cut-off, and the exponent

*κ*is typically between 0.9 and 1.20 [15–17]. Inclusion of such a model in Eq. (5) enables quantification of the expected imaging error of any hybrid imaging system that uses a linear restoration filter.

The following discussions assume that the image is sampled at greater than the Nyquist frequency so no aliasing occurs and that optimal performance is achievable by use of optimal restoration in which the aberration is known. That is; the kernel used in image restoration is optimally matched to the actual PSF for each level of aberration. This is appropriate when the aberration varies in a known way, e.g. with field position [18], or with zoom [19]. Even without *a priori* knowledge of the PSF, near-optimal image recovery is possible using blind-deconvolution techniques [20,21]. For third-order phase-modulations it has been shown that depth can be estimated accurately from defocus [22], thereby potentially increasing the convergence speed of blind-deconvolution algorithms. In general, it will therefore be feasible to approach the imaging performance assumed in the following analysis. We employ a circular optical aperture for calculations throughout this article.

## 3. Defocus tolerance with third-order phase-modulation

The importance of the amplitude of phase-modulation on attainable image quality is illustrated by the simulations shown in Fig. 1
, which depicts the variation of image error with defocus and amplitude of phase-modulation, *α*, for a pure-cubic phase function. The top row shows the error magnitudes of simulated images obtained with a hybrid imaging system incorporating a sub-optimal phase-modulation, *α* = ½*α*
_{opt}, for defocuses of *W*
_{20} = 0*λ*, *W*
_{20} = 3*λ*, and *W*
_{20} = 5*λ* subsequent to restoration with a Wiener filter. Τhe optimal phase-modulation, *α*
_{opt} = 2.87*λ*, is calculated using Eq. (4) for a defocus tolerance, |*W*
_{20}|*≤*5*λ*. Τhe error magnitudes are calculated with respect to the ideal, noiseless, diffraction-limited images. Error magnitudes for systems incorporating the optimal, and twice the optimal modulation depth are shown respectively on following two rows.

It can be appreciated that for small values of *α*, the suppression of the modulation-transfer function (MTF) for larger values of defocus introduces increasing error magnitudes in the recovered images (see for example the image for *α* = ½*α*
_{opt}, *W*
_{20} = 5*λ*). For increasing values of *α*, the errors become less sensitive to variations in *W*
_{20}; but for *α* ≥*α*
_{opt}, the errors averaged across the range of defocus is increased due to the increased suppression of the MTF by the pupil-plane phase-modulation. A large number of such simulations for representative scenes enables identification of the optimal *α* for these particular scenes, however, it is possible to obtain a more general result by assuming a model for the scene spatial spectrum, as suggested in section 2.

By way of example, we use Eq. (5) to estimate the expected imaging-error, *ε* with a signal-to-noise (SNR) model: *P _{S}(f)/P_{N}* = |10/

*f*|

^{2}, that is

*κ =*1, and with the standard deviation of the noise equal to 1/280 of the dynamic range. These are typical values for a well-illuminated, high dynamic range detector operating in detector-noise-limited conditions. In Fig. 2a

*ε*is plotted as a function of

*W*

_{20}for several values of

*α*. It can be appreciated from this figure that larger values of

*α*yield larger values of

*ε*for small

*W*

_{20}(due to increased suppression of the MTF). However,

*ε*is then less sensitive to defocus; hence for larger

*W*

_{20,}a larger

*α*yields a smaller

*ε*(by avoiding the zeros in the out-of-focus MTF of a traditional optical system). In applications for which a finite range of defocus up to a certain maximum value

*W*

_{20,max}is encountered, then$\overline{\epsilon}$, the quadratic mean of

*ε*over the range of

*W*

_{20}is a useful figure of merit. The solid-blue trace in Fig. 2b shows the variation of $\overline{\epsilon}$, averaged over the range −5

*λ*≤

*W*

_{20}≤ 5

*λ*as a function of

*α*. For this range of

*W*

_{20}, the optimal value of

*α*is found to be 2.87

*λ*.

It is interesting to compare the optimal values of $\overline{\epsilon}$obtained with the pure-cubic *(β*≡* 0* in Eq. (1)), the generalized cubic masks (*β*≈−3*α*), and the radial quartic profile given by Eq. (2). In Fig. 2b
$\overline{\epsilon}$ is plotted as a function of *α* for the former two, and in terms of *γ* for the latter. It can be seen that for these specific parameters minimum values of $\overline{\epsilon}$ are lowest for the cubic phase profile and are 12% higher for the generalized cubic and 30% higher for the radially-symmetric mask. Our simulations have shown that these differences are relatively insensitive to SNR and defocus range.

Using the general model, it is straightforward to predict the imaging fidelity of a hybrid imaging system with arbitrary phase- or amplitude-modulation. Here we apply our analysis to the generalized cubic phase-profiles described by Eq. (1) to determine the optimum values of *α* and *β* independently. The contour plot in Fig. 3a
shows the variation in imaging error, $\overline{\epsilon}$, as a function of *α* and *β* for a system with a defocus tolerance *|W*
_{20}
*| ≤* 5*λ*. A global optimum imaging fidelity is obtained near the axis *β* ≡ 0, and a secondary maximum can be noted on the axis *β* ≡ −3*α* in Fig. 3a.

Both optimal sets of (*α,β*) are plotted in Fig. 4a
as a function of the maximum value of defocus to be mitigated, *W*
_{20}
* _{,max}*. A quasi-linear increase of the global optimum of

*α*≈2

*W*

_{20,max}can be observed. This is in agreement with the analytically derived expression for the optimum value of

*α = (*1

*-f)*3

*W*

_{20}

*given in [23] when employing an effective spatial frequency*

_{,max}*f*= 1/3. These calculations also show that

*β*is optimal at approximately 0.3

*λ*, largely independent of

*W*

_{20}

*, although its effect on the performance is essential negligible.*

_{,max}Optimal sets of *α* and *β* for the secondary optimum are shown in Fig. 4b and have an approximately linear dependency with *W*
_{20,max}. Notice that the dependency between the two parameters closely follows *β≡-*3*α*, corresponding to a trefoil wave-front modulation. A very similar phase-modulation has been derived by minimization of the Fisher information in the PSF, while constraining reduction in the Strehl ratio [9,10].

In Fig. 4c the variation of $\overline{\epsilon}$ is shown with *W*
_{20,max} for both, approximately pure-cubic (*β* ≈0.3*λ*) and generalized cubic (*β*≈-3*α*) phase functions. It can be seen that $\overline{\epsilon}$increases monotonically with *W*
_{20,max} in both cases and that the approximately cubic function provides a slightly lower $\overline{\epsilon}$. The improvement offered by the approximately cubic mask is modest however and additional criteria may have a bearing on the preferred modulation function. For example, it has been found that high levels of astigmatism in a singlet lens were better mitigated using the generalized cubic phase profile [18].

## 4. Perception of hybrid imaging quality

To assess the relevance of these conclusions to perceived image quality, the mean structural similarity metric (MSSIM) has been used to assess image quality. This metric has been shown to correlate well with human perception of image similarity [13]. Instead of using a SNR-model, the metric is applied to a set of commonly used natural and synthetic test-images (Fig. 3c) modified by the corresponding OTFs for 21 defocus positions −5λ ≤ *W*
_{20} ≤ 5λ. Photon-noise, Gaussian sensor-noise, and analog-to-digital conversion at the detector are included in the simulation. The noisy images are then restored using the corresponding Wiener filter, compared to the theoretical diffraction-limited image using the MSSIM metric, and plotted in Fig. 3b. The MSSIM contour plot and the plot in Fig. 3a, obtained using the scene-model are very similar with almost identical optimal values of *α* and *β* indicating that the use of Eq. (4) as a metric of image quality yields a valid optimization of perceived image quality.

To illustrate the impact of the variation of image error with *W*
_{20}, Fig. 1 and Fig. 5
show false-coloring of the image errors for three defocus positions and two scenes: a spoke and a representative scene. Whereas Fig. 1 depicts image errors for the pure-cubic phase-modulation (*β*≡0), Fig. 5 depicts image errors for the generalized cubic with *β =* −3*α*, and quartic phase functions. Image errors are depicted for *W*
_{20} = 0, 3*λ*, and 5*λ* and for the amplitudes of phase-modulation equal to half the optimal values, the exact optimal values and twice the optimal values, where the optimal amplitudes are *α*
_{opt,gen} = 1.29*λ* for the generalized cubic and *γ*
_{opt} = 2.8*λ*, *δ*
_{opt} = −0.7 for the quartic mask.

As expected and as can be appreciated from these images, the lowest imaging error occurs for zero defocus and lower amplitudes of phase-modulation (images in first column and first and third rows), however for larger defocus (third column) the errors are larger. Similarly larger amplitudes of phase-modulation (third and sixth rows) yields less variation with *W*
_{20}, nevertheless the average values of the errors are larger. Note that only images for positive defocus are shown. The PSF of anti-symmetric phase-modulation varies as an even function of defocus; images for negative defocus are therefore identical to images for positive defocus. The same is not true for the quartic phase-modulation for which simulations were conducted for both positive and negative defocus. In this case however, the images shown are representative because the optimized *δ*-parameter in Eq. (2) shifts the region of low imaging error to coincide optimally with the defocus tolerance range −5λ ≤ *W*
_{20} ≤ 5λ.

In the images on row 2 of Fig. 1 it can be seen that the pure-cubic imaging error is relatively low for the horizontal and vertical spatial frequencies, this is directly related to the high MTF values of the pure-cubic along these axes. The same angular variation is not visible for the images of the generalized cubic phase function shown on row 2 of Fig. 5, however a modest hexagonal variation is still present. The quartic modulation, due to its radial symmetry, has an angle-independent imaging error.

## 5. Discussion

Even modest levels of aberration can suppress the MTF below levels that allow restoration of a high fidelity image. For instance, zeros occur in the MTF for *W*
_{20}>*λ*/2, leading to irretrievable contrast loss. The introduction of pupil-plane phase-modulation can significantly reduce the MTF variation, and thus increase the defocus-tolerance. We have shown however that the phase-modulation parameters should be chosen with care; noise amplification and regularization by the image restoration also reduce the expected image fidelity. The method described here allows identification of the phase-modulation parameters that maximize the expected imaging fidelity for a given defocus tolerance. The numerical phase-modulation parameters presented here can be used as initial values for optimization of specific and rigorously modeled hybrid designs.

The method described is insensitive to angular distribution of error. Nevertheless, it is to be expected that an image with angularly varying quality will be perceived differently from an image with an, on-average, uniformly distributed quality. This is pertinent for all antisymmetric masks, particularly the cubic phase-modulation which yields significantly lower errors for horizontal and vertical spatial frequencies. A potential improvement to the method presented here could be the incorporation of an angular weighting function to account for perceptual effects of the orientation of errors.

We have assumed here that the image is Nyquist sampled. Although many practical imaging systems are under-sampled, the magnitude of errors introduced by aliasing tend to be reduced by the phase-modulation function [24]. The effect of aliasing on the optimum pupil-phase-modulation is subject of further investigation.

Throughout this paper optimal performance has been calculated by the use of the optimal deconvolution kernel; that is, the kernel used in image recovery corresponds to the actual defocused PSF. In some applications it may be desirable or necessary to use a single kernel for a wide range of *W*
_{20} and this will introduce image errors that increase with defocus mismatch. In the case of the quartic phase-modulation, these errors are similar to those introduced by regularization and are well described by the MSSIM metric, but for antisymmetric phase profiles, such as the cubic and generalized cubic functions, artifacts are introduced that have the form of image replications [22]. An accurate metric of perceived quality of images manifesting this type of artifacts has yet to be defined; however, it is possible that for images for which it is not possible to use the optimal defocus in image recovery, the presence of these image-replication artifacts may yield lower perceived image quality for cubic and generalized cubic phase functions than for radially-symmetric phase functions.

## 6. Conclusion

Hybrid optical-digital imaging systems offer improved capabilities for reducing the effects of aberrations; however a new methodology is required to optimize imaging systems for output image quality. In this paper we introduce a method to predict the imaging fidelity of hybrid designs with realistic SNR ratios, thereby accounting for noise amplification and regularization of the image processing. The method can be used to define optimization metrics for any optical design for which the PSF can be determined, facilitating integration with existing commercial optical design software.

Using this technique, commonly studied antisymmetric and radially-symmetric phase functions have been analyzed and compared. Numerical simulations show that for a given depth-of-field range, the highest imaging fidelity can be obtained with only two combinations of *α* and *β* of the generalized cubic mask. Both configurations yield a lower minimum expected imaging error than what can be achieved with the quartic phase-modulation. The imaging fidelity was predicted using a general statistical signal-model, and the results are shown to be compatible with structural similarity evaluations on a set of test-scenes simulated with realistic levels of noise.

## Acknowledgments

We are grateful to Qioptiq, St. Asaph, UK and EPSRC for funding this research.

## References and links

**1. **E. R. Dowski Jr and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. **34**(11), 1859–1866 (1995). [CrossRef] [PubMed]

**2. **S. Mezouari and A. R. Harvey, “Combined amplitude and phase filters for increased tolerance to spherical aberration,” J. Mod. Opt. **50**(11), 2213–2220 (2003).

**3. **S. Mezouari and A. R. Harvey, “Phase pupil functions for reduction of defocus and spherical aberrations,” Opt. Lett. **28**(10), 771–773 (2003). [CrossRef] [PubMed]

**4. **S. Mezouari, G. Muyo, and A. R. Harvey, “Circularly symmetric phase filters for control of primary third-order aberrations: coma and astigmatism,” J. Opt. Soc. Am. A **23**(5), 1058–1062 (2006). [CrossRef]

**5. **K. S. Kubala, H. B. Wach, V. V. Chumachenko, and E. R. Dowski, “Increasing the depth of field in an LWIR system for improved object identification,” Proc. SPIE **5784**, 146–156 (2005). [CrossRef]

**6. **S. S. Sherif, W. T. Cathey, and E. R. Dowski, “Phase plate to extend the depth of field of incoherent hybrid imaging systems,” Appl. Opt. **43**(13), 2709–2721 (2004). [CrossRef] [PubMed]

**7. **Q. Yang, L. Liu, and J. Sun, “Optimized phase pupil masks for extended depth of field,” Opt. Commun. **272**(1), 56–66 (2007). [CrossRef]

**8. **N. Caron and Y. Sheng, “Polynomial phase masks for extending the depth of field of a microscope,” Appl. Opt. **47**(22), E39–E43 (2008). [CrossRef] [PubMed]

**9. **S. Prasad, T. C. Torgersen, V. P. Pauca, R. J. Plemmons, and J. van der Gracht, “Engineering the pupil phase to improve image quality,” Proc. SPIE **5108**, 1–12 (2003). [CrossRef]

**10. **S. Prasad, V. P. Pauca, R. J. Plemmons, T. C. Torgersen, and J. van der Gracht, “Pupil-phase optimization for extended-focus, aberration-corrected imaging systems,” Proc. SPIE **5559**, 335–345 (2004). [CrossRef]

**11. **W. Chi and N. George, “Electronic imaging using a logarithmic asphere,” Opt. Lett. **26**(12), 875–877 (2001). [CrossRef] [PubMed]

**12. **M. D. Robinson, G. Feng, and D. Stork, “Spherical coded imagers: improving lens speed, depth-of-field, and manufacturing yield through enhanced spherical aberration and compensating image processing,” Proc. SPIE **7429**, 74290M (2009). [CrossRef]

**13. **Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. **13**(4), 600–612 (2004). [CrossRef] [PubMed]

**14. **P. A. Jansson, *Deconvolution of images and spectra (2nd ed.)* (Ac. Press, Inc., Orlando, FL, USA, 1996).

**15. **D. L. Ruderman and W. Bialek, “Statistics of natural images: Scaling in the woods,” Phys. Rev. Lett. **73**(6), 814–817 (1994). [CrossRef] [PubMed]

**16. **E. P. Simoncelli, “Statistical Modeling of Photographic Images,” in *Handbook of Image and Video Processing*(Ac. Press, 2005), pp. 431–441.

**17. **A. van der Schaaf and J. H. van Hateren, “Modelling the power spectra of natural images: statistics and information,” Vision Res. **36**(17), 2759–2770 (1996). [CrossRef] [PubMed]

**18. **G. Muyo, A. Singh, M. Andersson, D. Huckridge, A. Wood, and A. R. Harvey, “Infrared imaging with a wavefront-coded singlet lens,” Opt. Express **17**(23), 21118–21123 (2009). [CrossRef] [PubMed]

**19. **M. Demenikov, E. Findlay, and A. R. Harvey, “Miniaturization of zoom lenses with a single moving element,” Opt. Express **17**(8), 6118–6127 (2009). [CrossRef] [PubMed]

**20. **J. Markham and J. A. Conchello, “Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur,” J. Opt. Soc. Am. A **16**(10), 2377–2391 (1999). [CrossRef]

**21. **Y. L. You and M. Kaveh, “Blind image restoration by anisotropic regularization,” IEEE Trans. Image Process. **8**(3), 396–407 (1999). [CrossRef] [PubMed]

**22. **M. Demenikov and A. R. Harvey, “A technique to remove image artefacts in optical systems with wavefront coding,” Proc. SPIE **7429**, 74290 (2009). [CrossRef]

**23. **G. Muyo and A. R. Harvey, “Decomposition of the optical transfer function: wavefront coding imaging systems,” Opt. Lett. **30**(20), 2715–2717 (2005). [CrossRef] [PubMed]

**24. **G. Muyo and A. R. Harvey, “The effect of detector sampling in wavefront-coded imaging systems,” J. Opt. A, Pure Appl. Opt. **11**(5), 054002 (2009). [CrossRef]