## Abstract

We present the first analytical analysis of image artifacts in defocused hybrid imaging systems that employ a cubic phase-modulation function. We show that defocus artifacts have the form of image replications and are caused by a net phase modulation of the optical transfer function. Both numerical simulations and experimental images are presented that exhibit replication artifacts that are compatible with the analytical expressions.

©2010 Optical Society of America

## 1. Introduction

In hybrid optical/digital systems the point-spread function (PSF) is modified with a phase-modulation function implemented in the aperture stop such that it is approximately insensitive to defocus and exhibits no nulls within the spatial-frequency pass-band. The in-focus PSF can then be used to restore a high quality image with an enhanced depth of field (DoF) [1–8]. A phase function with a cubic form [1,2] has, for example, been used to miniaturize zoom lenses with a single moving element [9], whilst a generalized-cubic phase function [3,4] has been used for mitigating field curvature and astigmatism [10]. The cubic phase-modulation function [1,2] is given by *z*(*x*,*y*) = *α*(*x*
^{3} + *y*
^{3}), where *α* is a constant that defines the optical path difference in units of wavelength *λ,* and *x* and *y* are normalized coordinates of the aperture stop. When there is a mismatch between the defocused PSF assumed in the image restoration and the actual defocused PSF, artifacts are observed in the restored image [3,4]. We show here that image artifacts are caused by a net modulation of the imaging system phase-transfer function (PTF) that varies non-linearly with spatial frequency. The system PTF refers to the net effect of the optical PTF and digital restoration PTF. This analysis enables expressions to be developed for the form of these artifacts and this opens up the possibility that an algorithm can be devised for their quantification which will enable the ideal image-restoration kernel to be deduced to yield an artifact-free image.

Several higher-order phase-functions with anti-symmetric phase profiles [3–6] introduce non-linear variations in optical PTF which produce image artifacts that vary from mask to mask [6]. Rotationally symmetric phase functions result in a purely real optical transfer function (OTF) and therefore introduce no image artifacts; the phase function in [6] was therefore made approximately rotational symmetric in order to reduce the non-linear phase variations. The disadvantage with the rotational symmetric phase masks, such as the radial quartic and logarithmic functions [11–13] is that, for a given Strehl ratio penalty, they give less extended depth-of-field (approximately half) compared to anti-symmetric phase masks, such as the purely cubic phase function [7].

In section 2, we present an analytical analysis of the introduction of image artifacts in a hybrid imaging system employing a cubic phase function. We restrict our analysis to the cubic form because its linear separability facilitates analytical treatment of the optical and system PTFs. In section 3, we numerically simulate imaging in a hybrid imaging system with a cubic phase mask and show that the image artifacts in the restored image are compatible with our analytical expressions. In section 4 we present image artifacts in experimentally restored images in a hybrid imaging system that are in agreement with analytical and numerical predictions and in section 5 we conclude and describe future work.

## 2. Analytical form of image artifacts in hybrid imaging systems

The one-dimensional OTF of a hybrid imaging system employing a cubic phase function can be found analytically using various methods [1,2]. Whereas derivation of the cubic phase profile neglected the phase of the OTF, subsequent analysis has shown that defocus introduces modest amplitude modulation and, more importantly, significant phase modulation in the OTF [2]. The phase of the OTF as a function of defocus, *W*
_{20}, is [2]:

*W*

_{20}is normalized with respect to

*λ*,

*ν*is spatial frequency normalized with respect to the cut-off frequency, and the maximum defocus for an approximately invariant modulation-transfer function (MTF) is |

*W*

_{20}|

_{max}= 3

*α*(1-

*v*) [2]. We assume in the following analysis that the image is recovered using a simple, commonly used inverse filter,

*F*= 1/

*H*

_{in}where

*H*

_{in}is the restoration OTF, typically in focus. The Fourier transform of the restored image is

*I*’

_{res}=

*F I*where

_{rec}*I*is the Fourier-transform of the recorded image. Since the MTF is approximately invariant to defocus it is common to use a single inverse filter to recover images over an extended range of defocus. We now show that this leads to replication-like artifacts. When other filters, such as for example a regularized Wiener filter are used, additional artifacts, such as ringing, or Gibbs phenomena, artifacts become apparent, but these are distinct from the phase-induced artifacts described here and can be reduced using iterative image restoration algorithms [14]. Optical convolution and post-detection deconvolution yields a phase-mismatch error, or system PTF of

_{rec}*Δθ*(

*ν*) =

*θ*(

*W*)−

_{20}*θ*(

*W*),), where

_{20,0}*θ*(

*W*) and

_{20}*θ*(

*W*),) are the convolution and deconvolution phase modulations respectively. In the absence of noise and with simple inverse restoration to a diffraction-limited, in-focus MTF, the restored image is described in the presence of defocus by:where

_{20,0}*I’*(

_{res}*ν*) and

*I*(

_{diff}*ν*) are the Fourier transforms of the restored image and the diffraction-limited image respectively. Amplitude modulation effects also exist [2] but associated errors in the image are at a much lower level and can be ignored. The constant term

*π*/4 in (1) is unimportant here and can be ignored. After some simple mathematical manipulation, the modulation of the system PTF can be described by:

*ΔW*

_{20}=

*W*

_{20}−|

*W*

_{20}|

_{max}and

*ΔW*

_{20,0}=

*W*

_{20,0}−|

*W*

_{20,0}|

_{max}. The first term is linear in

*ν*(due to a linear defocus dependent term in the OTF as described in [1,2]) and is responsible for translation in the image domain. The second term is a nonlinear phase shift responsible for image artifacts. When the restoration kernel is identical to the imaging PSF, that is when

*W*

_{20,0}=

*W*

_{20}, there is neither a linear image shift or image artifacts.

Our aim here is to understand the general form of the image artifacts to underpin and inform the development of a technique for their removal. We use several approximations, in addition to that used to derive (1), to show that the general character of the derived artifacts is compatible with those exhibited in numerical simulations and in experimental images. We omit the unimportant first term in (3) and observe that for higher frequencies, 0.7<*ν* <1, the multiplication factor in the second term varies by less than 20%. Applying the approximation *ν*≈1, and combining (2) and (3) yields:

*B*= 4π(Δ

*W*

_{20})

^{2}/(3α) and

*D*= 4π(Δ

*W*

_{20,0})

^{2}/(3α). Expression (4) is valid strictly only for

*ν*~1, but it is clear that a general expression can be derived for other specific values at

*ν*. Using the Bessel-function identity yields

*is the Bessel-function of first kind and order*

_{n}*n*, one obtains

*is the Bessel-function of first kind and order*

_{m}*m*and hence

The form of Eq. (7) is approximate, quite complex and does not readily lend itself to rigorous physical interpretation, however it can be seen that the restored image in the spatial domain is the original image (obtained with *n* = *m* = 0) superimposed with linearly translated and distorted replicas of the original image represented by the terms for *n*≠0, *m*≠0. It can be appreciated that since *ΔW*
_{20} varies with *ν*, so do *B* and *D* and hence the replicated images are distorted by the nonlinear *ν* dependence of the argument of the exponential term. It can also be appreciated that the translation of replicas will vary with *α*, *W*
_{20} and *W*
_{20,0} and will be zero for *W*
_{20} = *W*
_{20,0}.

## 3. Numerical demonstration of image artifacts in hybrid imaging systems

In this section we show using numerical simulations of restored images that image-replication-like artifacts compatible with Eq. (7) are evident. In the simulations, the cameraman image shown in Fig. 1(a)
is imaged using a cubic phase function with *α* = 5*λ* and with a defocus of *W*
_{20} = 2λ.The images in Figs. 1(b)-1(f) show the recovered image as the kernel used for the deconvolution is varied from *W*
_{20,0} = 0*λ* to *W*
_{20,0} = 4*λ*. It can be seen from Fig. 1(d) that when *W*
_{20,0} = *W*
_{20} = 2*λ,* the image is artifact free, but when *W*
_{20,0}≠*W*
_{20}, as shown in Figs. 1(b)-1(c) and Figs. 1(e)-1(f) the recovered images are significantly degraded by image artifacts.

Similarly, if the *W*
_{20,0} = 0*λ* kernel is employed to recover all images, artifacts are also evident in all except the *W*
_{20} = 0*λ* recovered image, however, the images in Fig. 1, indicate that the artifacts vary irregularly with *W*
_{20,0}-*W*
_{20} and that their form is asymmetric about *W*
_{20,0}-*W*
_{20} = 0. The artifacts have the form of translated image replications, compatible with Eq. (7) and the nonlinear phase variation has introduced a high-pass filter characteristic that varies in a complicated way with defocus. The images in Fig. 2
were obtained for *W*
_{20,0} = 0 and *W*
_{20} = 1*λ*, for *α* = 2.5*λ*, 5*λ* and 10λ. It can be seen that the spatial separation of the replications increases with *α*, compatible with Eq. (7).

## 4. Experimental demonstration of image artifacts in hybrid imaging systems

In this section we present experimental images that exhibit replication artifacts. Whilst the cameraman image used in Fig. 2 was selected to show clearly the form of replication artifacts, we describe here the imaging of a color target that lucidly demonstrates replication artifacts associated with defocus of a practical system.

Images were recorded with a hybrid imaging system employing cubic phase-functions implemented as discrete phase masks integrated into the aperture stop of a Nikon AF Nikkor 50mm f/1.8D lens. The phase masks were manufactured by laser-polishing of fused silica plates with an active diameter of 6 mm. Images are presented for *α* = 0*λ*, 5*λ* and 10*λ* (*λ* = 550nm). Images were recorded using a Nikon D40X camera. The images in Fig. 3
are of a dart board at a distance of 1.5m recorded with varying degrees of defocus and values of α. The images in the top row were recorded with α = 0 (corresponding to a conventional imaging system) with the lens focused at a distance of 1.0m, corresponding to W_{20}≈–3λ (left), at 1.5m, corresponding to W_{20} = 0 (centre) and at infinity, implying W_{20}≈8.6λ (right); similarly the images for α = 5λ and α = 10λ are shown in the second and third rows respectively. All images in the second and third rows were restored using the appropriate kernels representing the PSF when the lens is focused at 1.5m.

The recovered images in the second and third rows clearly show better and more constant image sharpness for both in- and out-of-focus distances compared to the images in the first row recorded with the conventional system without. However, significant variations in the system PTF occur for objects displaced from the nominal in-focus range of 1.5m and as expected from the Eq. (7), these yield the image replication artifacts apparent in Figs. 3(d), 3(f), 3(g), 3(i)). The restored images in Figs. 3(e) and 3(h) should in principle be free of image artifacts since the use of the correct restoration kernel yields negligible modulation of the PTF according to Eq. (7). In practice however small differences between the restoration kernel and the imaging PSF, due to experimental recording errors and noise removal, have resulted in low level image artifacts that are visible in Figs. 3(e) and 3(h). By comparison with the images recorded with *α* = 0, the improved constancy of image quality is accompanied by a general reduction in signal-to-noise-ratio due to the suppressed MTF in the recorded image.

It is interesting to observe that, although a larger value of *α* yields a better invariance of modulation transfer function (MTF), a consideration of overall image quality may favor a smaller value of *α*. First of all, a strong phase-modulation introduces larger noise amplification in the restored image, and secondly it introduces image replicas that are more displaced than those introduced by a weaker phase mask. Incorporation of these issues into image quality optimization algorithms is vital if the potential benefit of hybrid imaging is to be realized; that is, the modulation transfer function alone is not sufficient.

## 5. Conclusions and future work

We have shown for the first time how imaging artifacts in hybrid imaging systems are associated with modulation of the phase-transfer function and we have shown that for the cubic phase-function, these artifacts have the form of image replications. We have shown both analytically, theoretically and experimentally that image artifacts are absent in the restored image only when the restoration kernel matches the imaging PSF. Optimal exploitation of the hybrid imaging concept will require techniques to avoid the presence of these artifacts; for example, algorithms that are able to deduce the imaging OTF from a quantification of the artifacts described here will enable aberration-free images to be recovered without *a priori* knowledge of the defocus. Since the image artifacts are induced by phase mismatches in image recovery, they are not affected by the noise in the recorded image, however, attempts to attenuate the artifacts by deduction of the imaging OTF will at some point be limited by noise-reduced accuracy in the OTF estimation.

## Acknowledgement

We would like to thank Scottish Enterprise for funding, Ewan Findlay at STMicroelectronics for technical support and Gonzalo Muyo for helpful comments.

## References and links

**1. **J. E. R. Dowski Jr and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. **34**(11), 1859 (1995). [CrossRef] [PubMed]

**2. **G. Muyo and A. R. Harvey, “Decomposition of the optical transfer function: wavefront coding imaging systems,” Opt. Lett. **30**(20), 2715–2717 (2005). [CrossRef] [PubMed]

**3. **S. Prasad, T. C. Torgersen, V. P. Pauca, R. J. Plemmons, and J. van der Gratch, “Engineering the pupil phase to improve image quality,” Proc. SPIE **5108**, 1–12 (2003). [CrossRef]

**4. **S. Prasad, V. P. Pauca, R. J. Plemmons, T. C. Torgersen, and J. van der Gracht, “Pupil-phase optimization or extended focus, aberration corrected imaging systems,” Proc. SPIE **5559**, 335–345 (2004). [CrossRef]

**5. **D. S. Barwick, “Efficient metric for pupil-phase engineering,” Appl. Opt. **46**(29), 7258–7261 (2007). [CrossRef] [PubMed]

**6. **D. S. Barwick, “Increasing the information acquisition volume in iris recognition systems,” Appl. Opt. **47**(26), 4684–4691 (2008). [CrossRef] [PubMed]

**7. **P. Mouroulis, “Depth of field extension with spherical optics,” Opt. Express **16**(17), 12995–13004 (2008). [CrossRef] [PubMed]

**8. **G. Muyo and A. R. Harvey, “The effect of detector sampling in wavefront-coded imaging systems,” J. Opt. A, Pure Appl. Opt. **11**(5), 054002–054010 (2009). [CrossRef]

**9. **M. Demenikov, E. Findlay, and A. R. Harvey, “Miniaturization of zoom lenses with a single moving element,” Opt. Express **17**(8), 6118–6127 (2009). [CrossRef] [PubMed]

**10. **G. Muyo, A. Singh, M. Andersson, D. Huckridge, A. Wood, and A. R. Harvey, “Infrared imaging with a wavefront-coded singlet lens,” Opt. Express **17**(23), 21118–21123 (2009). [CrossRef] [PubMed]

**11. **S. Mezouari and A. R. Harvey, “Phase pupil functions for reduction of defocus and spherical aberrations,” Opt. Lett. **28**(10), 771–773 (2003). [CrossRef] [PubMed]

**12. **N. George and W. Chi, “Extended depth of field using a logarithmic asphere,” J. Opt. A, Pure Appl. Opt. **5**(5), S157–S163 (2003). [CrossRef]

**13. **S. Mezouari, G. Muyo, and A. Harvey, “Circularly symmetric phase filters for control of primary third-order aberrations: coma and astigmatism,” J. Opt. Soc. Am. A **23**(5), 1058–1062 (2006). [CrossRef]

**14. **J. van der Gracht, J. Nagy, V. Pauca, and R. Plemmons, “Iterative restoration of wavefront coded imagery for focus invariance,” in Integrated Computational Imaging Systems, OSA Technical Digest Series (Optical Society of America, 2001).