1. Introduction

The use of pupil-plane phase-modulation in hybrid optical-digital imaging systems can enable good imaging quality even in the presence of high levels of optical aberration, such as defocus, astigmatism and spherical aberration [1–4]. We use the term hybrid imaging to refer to a system that employs a modification of the point-spread function (PSF) to optically encode an image together with digital decoding. This process of optical encoding and digital decoding is analogous to the use of a codec to enhance transmission of a signal through an imperfect telecommunication channel: in the case of hybrid imaging, the aim is to reduce sensitivity to aberrations in the optics that would otherwise reduce image quality. In earlier work a hybrid imaging system was designed for improved object identification [5]. In this paper we introduce a method to find the phase-profile and modulation depth that maximize the expected image fidelity for a required aberration tolerance. Crucial for such a hybrid design is a well-considered choice of phase-profile used for optical encoding. We describe here the optimization of the most important phase-modulations for reduced sensitivity to defocus; namely third-order antisymmetric profiles and radially-symmetric quartic phase-modulation. However the technique is more generally applicable to higher order aberrations, as well as to other phase- or amplitude-modulations.

Various types of phase-modulations have been shown to increase tolerance to defocus; one of the most common is the cubic phase-modulation and its generalizations of the form:

Also of interest are radially-symmetric phase profiles such as the logarithmic asphere [11], the quartic phase-mask [3,4] and spherical coding [12]. The latter two can be described as:

In the next section we describe how maximum-likelihood image restoration can be used to estimate the expected imaging-error due to aberration, noise, and image restoration in hybrid imaging systems. In section 3 we evaluate and compare imaging performance for phase-profiles described by Eq. (1) and Eq. (2). For the former class of phase profiles, two regimes of third-order phase-profiles are found to yield optimal imaging performance, and we show that for a given focal-depth extension, an optimal balance exists between defocus-tolerance and the reduction in signal-to-noise ratio due to suppression of the modulation-transfer function (MTF) by the pupil-plane phase-modulation. The imaging performance of these third-order anti-symmetric optima is compared to that of the radially-symmetric quartic phase-modulation. Finally, in section 4 the validity of the model is tested on a set of scenes using a structural-similarity image metric that has shown good agreement with the perceived similarity of images in psychophysical experiments [13].

2. Hybrid imaging fidelity evaluation

A hybrid imaging system can be considered to perform well if the final restored image is similar to the ideal noise-free, diffraction-limited image. To quantify performance, the expected mean-squared error between the ideal and restored images is calculated over the required aberration tolerance range. We define the imaging fidelity here as the inverse of the expected mean-squared error. This mathematically convenient definition allows the use of a general scene- and noise-model, enabling efficient numerical optimization.

One can write the expected mean-squared error, ${\overline{\epsilon }}^2$of a shift-invariant system as a function of the Fourier-transform of the restored image, I_r, and the noiseless diffraction-limited image, I_dl:

The following discussions assume that the image is sampled at greater than the Nyquist frequency so no aliasing occurs and that optimal performance is achievable by use of optimal restoration in which the aberration is known. That is; the kernel used in image restoration is optimally matched to the actual PSF for each level of aberration. This is appropriate when the aberration varies in a known way, e.g. with field position [18], or with zoom [19]. Even without a priori knowledge of the PSF, near-optimal image recovery is possible using blind-deconvolution techniques [20,21]. For third-order phase-modulations it has been shown that depth can be estimated accurately from defocus [22], thereby potentially increasing the convergence speed of blind-deconvolution algorithms. In general, it will therefore be feasible to approach the imaging performance assumed in the following analysis. We employ a circular optical aperture for calculations throughout this article.

3. Defocus tolerance with third-order phase-modulation

The importance of the amplitude of phase-modulation on attainable image quality is illustrated by the simulations shown in Fig. 1 , which depicts the variation of image error with defocus and amplitude of phase-modulation, α, for a pure-cubic phase function. The top row shows the error magnitudes of simulated images obtained with a hybrid imaging system incorporating a sub-optimal phase-modulation, α = ½α _opt, for defocuses of W ₂₀ = 0λ, W ₂₀ = 3λ, and W ₂₀ = 5λ subsequent to restoration with a Wiener filter. Τhe optimal phase-modulation, α _opt = 2.87λ, is calculated using Eq. (4) for a defocus tolerance, |W ₂₀|≤5λ. Τhe error magnitudes are calculated with respect to the ideal, noiseless, diffraction-limited images. Error magnitudes for systems incorporating the optimal, and twice the optimal modulation depth are shown respectively on following two rows.

 

Fig. 1 False-color coding of error magnitudes of simulated images recorded with pure-cubic pupil phase-modulation (β ≡ 0) for various values of α and W ₂₀. The colors indicate pixel value differences between the restored image and the ideal, noise-less diffraction-limited image, normalized to the dynamic range. The optimal modulation depth, α _opt,cubic of the pure-cubic for a defocus tolerance |W ₂₀| ≤ 5λ is 2.87λ. The two left-most images in Fig. 3c, of the spoke target and the cameraman, are used as test-scenes.

Download Full Size | PDF

It can be appreciated that for small values of α, the suppression of the modulation-transfer function (MTF) for larger values of defocus introduces increasing error magnitudes in the recovered images (see for example the image for α = ½α _opt, W ₂₀ = 5λ). For increasing values of α, the errors become less sensitive to variations in W ₂₀; but for α ≥α _opt, the errors averaged across the range of defocus is increased due to the increased suppression of the MTF by the pupil-plane phase-modulation. A large number of such simulations for representative scenes enables identification of the optimal α for these particular scenes, however, it is possible to obtain a more general result by assuming a model for the scene spatial spectrum, as suggested in section 2.

By way of example, we use Eq. (5) to estimate the expected imaging-error, ε with a signal-to-noise (SNR) model: P_S(f)/P_N = |10/f|², that is κ = 1, and with the standard deviation of the noise equal to 1/280 of the dynamic range. These are typical values for a well-illuminated, high dynamic range detector operating in detector-noise-limited conditions. In Fig. 2a ε is plotted as a function of W ₂₀ for several values of α. It can be appreciated from this figure that larger values of α yield larger values of ε for small W ₂₀ (due to increased suppression of the MTF). However, ε is then less sensitive to defocus; hence for larger W _20, a larger α yields a smaller ε (by avoiding the zeros in the out-of-focus MTF of a traditional optical system). In applications for which a finite range of defocus up to a certain maximum value W _20,max is encountered, then$\overline{\epsilon }$, the quadratic mean of ε over the range of W ₂₀ is a useful figure of merit. The solid-blue trace in Fig. 2b shows the variation of $\overline{\epsilon }$, averaged over the range −5λ ≤ W ₂₀ ≤ 5λ as a function of α. For this range of W ₂₀, the optimal value of α is found to be 2.87λ.

 

Fig. 2 (a) The square root of the expected mean-square imaging-error normalized to the dynamic range, ε, is shown as a function of defocus for various cubic phase-modulation depths α; (b) shows in solid-blue line the average imaging-error as a function of α for |W ₂₀ | ≤ 5λ. For comparison, the dashed-green line and dot-dashed-red line show the average imaging error of respectively the generalized cubic mask with β = −3α, and the quartic phase-modulation with optimal δ = −0.7, as a function of their respective parameters α and γ.

Download Full Size | PDF

It is interesting to compare the optimal values of $\overline{\epsilon }$obtained with the pure-cubic (β≡ 0 in Eq. (1)), the generalized cubic masks (β≈−3α), and the radial quartic profile given by Eq. (2). In Fig. 2b $\overline{\epsilon }$ is plotted as a function of α for the former two, and in terms of γ for the latter. It can be seen that for these specific parameters minimum values of $\overline{\epsilon }$ are lowest for the cubic phase profile and are 12% higher for the generalized cubic and 30% higher for the radially-symmetric mask. Our simulations have shown that these differences are relatively insensitive to SNR and defocus range.

Using the general model, it is straightforward to predict the imaging fidelity of a hybrid imaging system with arbitrary phase- or amplitude-modulation. Here we apply our analysis to the generalized cubic phase-profiles described by Eq. (1) to determine the optimum values of α and β independently. The contour plot in Fig. 3a shows the variation in imaging error, $\overline{\epsilon }$, as a function of α and β for a system with a defocus tolerance |W ₂₀ | ≤ 5λ. A global optimum imaging fidelity is obtained near the axis β ≡ 0, and a secondary maximum can be noted on the axis β ≡ −3α in Fig. 3a.

 

Fig. 3 (a) $\overline{\epsilon }$ [dB] for generalized cubic phase-masks as described by Eq. (1) calculated using a statistical model of the scene. (b) The structural similarity of the same hybrid imaging systems averaged for the test-scenes shown in c), see section 5 for further discussion.

Download Full Size | PDF

Both optimal sets of (α,β) are plotted in Fig. 4a as a function of the maximum value of defocus to be mitigated, W _{20 ,max}. A quasi-linear increase of the global optimum of α ≈2W _20,max can be observed. This is in agreement with the analytically derived expression for the optimum value of α = (1-f)3W _{20 ,max} given in [23] when employing an effective spatial frequency f = 1/3. These calculations also show that β is optimal at approximately 0.3λ, largely independent of W _{20 ,max}, although its effect on the performance is essential negligible.

 

Fig. 4 (a) The global optimal phase-modulation parameters α and β as a function of the required defocus tolerance. (b) The secondary optimal phase-modulation parameters as a function of the required defocus tolerance. (c) The root of the expected mean-square imaging-error for both profile types as function of the maximum defocus W _{20 ,max}.

Download Full Size | PDF

Optimal sets of α and β for the secondary optimum are shown in Fig. 4b and have an approximately linear dependency with W _20,max. Notice that the dependency between the two parameters closely follows β≡-3α, corresponding to a trefoil wave-front modulation. A very similar phase-modulation has been derived by minimization of the Fisher information in the PSF, while constraining reduction in the Strehl ratio [9,10].

In Fig. 4c the variation of $\overline{\epsilon }$ is shown with W _20,max for both, approximately pure-cubic (β ≈0.3λ) and generalized cubic (β≈-3α) phase functions. It can be seen that $\overline{\epsilon }$increases monotonically with W _20,max in both cases and that the approximately cubic function provides a slightly lower $\overline{\epsilon }$. The improvement offered by the approximately cubic mask is modest however and additional criteria may have a bearing on the preferred modulation function. For example, it has been found that high levels of astigmatism in a singlet lens were better mitigated using the generalized cubic phase profile [18].

4. Perception of hybrid imaging quality

To assess the relevance of these conclusions to perceived image quality, the mean structural similarity metric (MSSIM) has been used to assess image quality. This metric has been shown to correlate well with human perception of image similarity [13]. Instead of using a SNR-model, the metric is applied to a set of commonly used natural and synthetic test-images (Fig. 3c) modified by the corresponding OTFs for 21 defocus positions −5λ ≤ W ₂₀ ≤ 5λ. Photon-noise, Gaussian sensor-noise, and analog-to-digital conversion at the detector are included in the simulation. The noisy images are then restored using the corresponding Wiener filter, compared to the theoretical diffraction-limited image using the MSSIM metric, and plotted in Fig. 3b. The MSSIM contour plot and the plot in Fig. 3a, obtained using the scene-model are very similar with almost identical optimal values of α and β indicating that the use of Eq. (4) as a metric of image quality yields a valid optimization of perceived image quality.

To illustrate the impact of the variation of image error with W ₂₀, Fig. 1 and Fig. 5 show false-coloring of the image errors for three defocus positions and two scenes: a spoke and a representative scene. Whereas Fig. 1 depicts image errors for the pure-cubic phase-modulation (β≡0), Fig. 5 depicts image errors for the generalized cubic with β = −3α, and quartic phase functions. Image errors are depicted for W ₂₀ = 0, 3λ, and 5λ and for the amplitudes of phase-modulation equal to half the optimal values, the exact optimal values and twice the optimal values, where the optimal amplitudes are α _opt,gen = 1.29λ for the generalized cubic and γ _opt = 2.8λ, δ _opt = −0.7 for the quartic mask.

 

Fig. 5 False-color hybrid imaging error for various phase-modulations and defocus. The secondary optimum and multiples of the generalized cubic phase-modulation (α _opt,gen = 1.29λ, β = −3α), are used for the error magnitudes shown in the top three rows. The quartic with optimal parameters γ _opt = 2.8λ and δ _opt = −0.7 and multiples are used for the error magnitudes in the bottom three rows.

Download Full Size | PDF

As expected and as can be appreciated from these images, the lowest imaging error occurs for zero defocus and lower amplitudes of phase-modulation (images in first column and first and third rows), however for larger defocus (third column) the errors are larger. Similarly larger amplitudes of phase-modulation (third and sixth rows) yields less variation with W ₂₀, nevertheless the average values of the errors are larger. Note that only images for positive defocus are shown. The PSF of anti-symmetric phase-modulation varies as an even function of defocus; images for negative defocus are therefore identical to images for positive defocus. The same is not true for the quartic phase-modulation for which simulations were conducted for both positive and negative defocus. In this case however, the images shown are representative because the optimized δ-parameter in Eq. (2) shifts the region of low imaging error to coincide optimally with the defocus tolerance range −5λ ≤ W ₂₀ ≤ 5λ.

In the images on row 2 of Fig. 1 it can be seen that the pure-cubic imaging error is relatively low for the horizontal and vertical spatial frequencies, this is directly related to the high MTF values of the pure-cubic along these axes. The same angular variation is not visible for the images of the generalized cubic phase function shown on row 2 of Fig. 5, however a modest hexagonal variation is still present. The quartic modulation, due to its radial symmetry, has an angle-independent imaging error.

5. Discussion

Even modest levels of aberration can suppress the MTF below levels that allow restoration of a high fidelity image. For instance, zeros occur in the MTF for W ₂₀>λ/2, leading to irretrievable contrast loss. The introduction of pupil-plane phase-modulation can significantly reduce the MTF variation, and thus increase the defocus-tolerance. We have shown however that the phase-modulation parameters should be chosen with care; noise amplification and regularization by the image restoration also reduce the expected image fidelity. The method described here allows identification of the phase-modulation parameters that maximize the expected imaging fidelity for a given defocus tolerance. The numerical phase-modulation parameters presented here can be used as initial values for optimization of specific and rigorously modeled hybrid designs.

The method described is insensitive to angular distribution of error. Nevertheless, it is to be expected that an image with angularly varying quality will be perceived differently from an image with an, on-average, uniformly distributed quality. This is pertinent for all antisymmetric masks, particularly the cubic phase-modulation which yields significantly lower errors for horizontal and vertical spatial frequencies. A potential improvement to the method presented here could be the incorporation of an angular weighting function to account for perceptual effects of the orientation of errors.

We have assumed here that the image is Nyquist sampled. Although many practical imaging systems are under-sampled, the magnitude of errors introduced by aliasing tend to be reduced by the phase-modulation function [24]. The effect of aliasing on the optimum pupil-phase-modulation is subject of further investigation.

Throughout this paper optimal performance has been calculated by the use of the optimal deconvolution kernel; that is, the kernel used in image recovery corresponds to the actual defocused PSF. In some applications it may be desirable or necessary to use a single kernel for a wide range of W ₂₀ and this will introduce image errors that increase with defocus mismatch. In the case of the quartic phase-modulation, these errors are similar to those introduced by regularization and are well described by the MSSIM metric, but for antisymmetric phase profiles, such as the cubic and generalized cubic functions, artifacts are introduced that have the form of image replications [22]. An accurate metric of perceived quality of images manifesting this type of artifacts has yet to be defined; however, it is possible that for images for which it is not possible to use the optimal defocus in image recovery, the presence of these image-replication artifacts may yield lower perceived image quality for cubic and generalized cubic phase functions than for radially-symmetric phase functions.

6. Conclusion

Hybrid optical-digital imaging systems offer improved capabilities for reducing the effects of aberrations; however a new methodology is required to optimize imaging systems for output image quality. In this paper we introduce a method to predict the imaging fidelity of hybrid designs with realistic SNR ratios, thereby accounting for noise amplification and regularization of the image processing. The method can be used to define optimization metrics for any optical design for which the PSF can be determined, facilitating integration with existing commercial optical design software.

Using this technique, commonly studied antisymmetric and radially-symmetric phase functions have been analyzed and compared. Numerical simulations show that for a given depth-of-field range, the highest imaging fidelity can be obtained with only two combinations of α and β of the generalized cubic mask. Both configurations yield a lower minimum expected imaging error than what can be achieved with the quartic phase-modulation. The imaging fidelity was predicted using a general statistical signal-model, and the results are shown to be compatible with structural similarity evaluations on a set of test-scenes simulated with realistic levels of noise.