A new methodology for image sensor modulation transfer function measurement using band-limited laser speckle is presented. We use a circular opal milk glass diffuser illuminated by a 5mW He-Ne laser and a linear polarizer to generate band-limited speckle on the sensor. The power spectral density cut-off frequency of the speckle is chosen to be twice that of the sensor Nyquist frequency by placing the sensor at the specific Z location along the optical axis. For the speckle input, we calculate the power spectral density at the sensor using the Rayleigh-Sommerfeld integral and then measure the output power spectral density for the speckle pattern captured by the sensor. With these data, the two-dimensional image sensor modulation transfer function (MTF) is calculated.
© 2008 Optical Society of America
CMOS image sensors are widely used on digital imaging devices. The modulation transfer function (MTF) of the image sensors is an important characteristic to evaluate the overall imaging system quality. The image sensor MTF reflects the spatial frequency response of the sensor. It is determined by the spatial structure and optical and electrical cross-talk of the pixels.
There have been several techniques widely used for measuring sensor MTF. For example, the sensor MTF is measured with bar targets , slanted edge technique , random targets , self-calibrating fringe pattern , and laser speckle modulated with a double-slit aperture . Among these methods, the slanted edge method is commonly applied in industry. A super resolution scan can be created with a slanted edge , which solves the sample-scene phase problem . The random targets method also solves the sample-scene phase problem. However using either a slanted edge or random targets requires a high quality lens to image the targets or patterns on the sensor. This need for a high quality lens greatly limits the application to image sensors with pixel size less than 2.0µm. The laser speckle modulated with a double-slit aperture provides MTF value at one single frequency for each single measurement as well. Also, only one dimensional MTF data depending on the direction of slit apertures is measured.
In this paper we present a new method for image sensor MTF measurements using band-limited laser speckle. The setup is simpler compared to other methods and it is particularly well-suited when the pixels are small. There are no lenses or slit apertures used in the setup. The advantage inherent in the use of a speckle pattern for this input is that it contains an excellent range of spatial frequencies from low to very high values. The limiting spatial frequency on the high side is given approximately by in the setup of Fig. 1. The two-dimensional sensor MTF data are obtained with each measurement. The method is applicable to sensors with small pixels. In Section 2, we present the theory we used to obtain the sensor MTF using band-limited laser speckle. The way to generate and calculate the speckle pattern and power spectral density is described. In Section 3, the experimental results for a 6.0µm monochrome CMOS sensor and a 2.2µm monochrome CMOS sensor are presented. The calculated input power spectral density is also compared with the measured data. The measured sensor MTF using our band-limited laser speckle method is compared with the data obtained using the standard slanted edge technique. The two methods provide results in excellent agreement, which indicates applicability of our developed methodology to characterization of modern CMOS image sensors with small pixel size.
2. Theory of measuring sensor MTF using band-limited laser speckle
Here we describe our methodology of measuring sensor MTF using laser speckle generated by a circular opal milk glass diffuser and a linear polarizer. The speckle theory involved in this method is explained. The power spectral density (PSD) of the speckle pattern on the sensor plane is calculated. The aliasing artifact involved in the sensor MTF measurement is described. The method to avoid aliasing artifact is also presented.
2.1. Speckle theory for sensor MTF measurement
We consider the linearly polarized speckle patterns. In our experimental setup, it is necessary to use a linear polarizer after the opal milk glass (OMG) diffuser in order to obtain a linearly polarized laser speckle pattern. The intensity of the speckle at the input to the sensor plane is a random process I (x, y)with coordinates (x,y) at the sensor plane. The output signal from the sensor is denoted as S(x,y). If the response of the sensor is linear and the system is shift-invariant, then the output signal S(x,y) can be expressed as:
where h is the impulse response for the image sensor. As explained in , the sensor array is not a spatially shift-invariant system. The response is dependent on the relative location of the point source and pixel center. By using the slanted edge or random targets or laser speckle, the sample-scene phases are averaged. With the averaging of the sample-scene phases and the use of speckle pattern, the sensor array becomes spatially shift-invariant as described by Eq. (1).
It is well-known that the ensemble-averaged autocorrelation function of the output signal for a spatially shift-invariant linear system is related to the ensemble-averaged autocorrelation function of the input random signal as:
where Rs (Δx,Δy) is the ensemble-averaged autocorrelation function of the captured speckle intensity pattern; RI(Δx,Δy) is the ensemble-averaged autocorrelation function of the input speckle intensity pattern; h is the impulse response function of the image sensor. We Fourier transform Eq. (2), then we obtain
where PSDS(fx,fy) is the power spectral density (PSD) of the speckle intensity captured by the CMOS sensor; PSDI(fx,fy) is the PSD of the input speckle intensity on the sensor plane; MTF is the modulation transfer function for the sensor. Eq. (3) is consistent with the published literature . To measure this sensor MTF, we need to measure the PSDS(fx, fy) of the speckle pattern captured by CMOS sensor and calculate the input PSDI(fx,fy) of the speckle on the sensor plane.
One way to obtain the PSD of the input speckle on the sensor plane is to calculate the scalar electric field on the sensor plane. The experimental setup is shown in Fig. 1. The OMG diffuser has a diameter of D=25mm at the input aperture plane. The distance between this aperture plane and the sensor plane is Z. A 5mW He-Ne laser is used for illumination. The collimated laser light illuminates the OMG diffuser. A linear polarizer is placed behind the OMG diffuser so that only a scalar electric field is considered here. In the following paragraphs we describe the diffuser and the calculation of the intensity I(x,y) received at the CMOS detector.
For creating a fully developed speckle pattern at the CMOS detector, it is well-known that one needs to use a volume-type of thick diffuser so that no specular beam is present. Opal milk glass (300 to 700µ m thickness) is an ideal choice for this, since from electron microscope studies it is known to consist of tiny (0.1µ m) spheres of higher index material created by thermally cycling glass around the annealing temperature. The output radiation for an input laser beam is fully depolarized with electric field spatial fluctuations in the exiting plane approaching the theoretical limit of λ/2, see . Alternatively, one can use an integrating sphere instead of OMG since the 100 or so bounces produces a fine source for speckle experiments. However, we have found that the stability to mechanical vibrations of the OMG provides a superior result.
Interesting too, while the field variations at the output of a 25mm OMG may be on the order of 109 in numerical calculations of a fully developed speckle pattern, one can obtain excellent results taking only 104 random points, as we described below .
We first generate the scalar electric field E0(x0,y0) on the aperture plane. For visible light or laser light, the OMG is like an ideal black body. The output radiation is completely unpolarized and fully speckled. Therefore, the phase of the linearly polarized electric field along the y-axis, E0(x0,y0), on the aperture plane is a uniformly distributed random variable. Let us assume the electric field E0(x0,y0) as:
where A(x0,y0) is the amplitude of the electric field. A(x0,y0) is a Gaussian function of (x0,y0) measured with a power meter. It is expressed as
where A(x0,y0) is normalized at the center of the diffuser; D=25mm which is the diameter size of the OMG diffuser. With the power meter, we measure the laser illumination intensity at the edge of the OMG diffuser is one fifth of that at the center of the diffuser. Phase θA(x0,y0) is a random variable with probability density function of
With the random field on the aperture plane given by Eqs. (4) to (6), we can calculate the scalar electric field Ey(x,y) on the sensor plane which is located at Z away from the aperture plane. We calculate the scalar electric field on the sensor plane using the Rayleigh-Sommerfeld integral which is the exact solution to Maxwell equation with the evanescent waves neglected, as shown below:
where Ey(x,y,Z) is the scalar electric field on the sensor plane, Z is the distance between the aperture plane at z=0 and sensor plane at z=Zand λ is the illumination laser light wavelength. With Eqs. (4) to (7), we calculate the speckle intensity pattern I(x,y,Z) which is the absolute square of Ey(x,y,Z,). The laser illumination intensity on the diffuser has to be measured accurately for an accurate calculation of Ey(x,y,Z). The reason that we numerically calculate PSD of the input speckle pattern on the sensor instead of treating PSDI(fx,fy) as the auto-correlation function of the amplitude of the scalar electric filed on the aperture plane is the Fresnel-zone approximation of Eq. (7) is not valid anymore when Z is comparable with the diffuser size and sensor size. As a part of the numerical simulation of the delta function sources on the opal milk glass, we also include a random position of this grid by means of uniformly distributed random points (x0,y0). The integration of Eq. (7) is replaced by summing the kernel of the integral over the aperture plane numerically. To calculate Ey(x,y,Z) accurately, we sample the aperture plane at z=0 with 100 by 100 random points. While the actual number of independent delta-like sources at the exit plane of the diffuser may be on the order of 109, we show by direct computer studies that results of about 1% accuracy can be attained using this sampling of 100 by 100 points.
The calculated speckle pattern with λ=0.6328µm, D=25mm, Z=87mm is shown in Fig. 2.
The transverse coherence length dt of the speckle is 2.2µm. dt is interpreted as the variation needed to decorrelate the auto-correlation function of the speckle. It is known  that the transverse speckle size dt is given by
The size of the speckle t d is chosen as the pixel size of the sensor by positioning the sensor with the right distance Z from the OMG diffuser. To measure the MTF of the sensor with pixel size of P, we locate the sensor at z=Z which is given by
Hence, we obtain Z=87mm for pixel size of 2.2µm. With this choice of Z, the speckle has a PSD that is high enough at the Nyquist frequency of the sensor and becomes 0 at double the Nyquist frequency. This can be seen in Fig. 3.
We calculate the PSD(fx,fy) of the speckle from the definition of PSD  which is given by
where X,Y are the overall size of the speckle pattern along the x and y axes, respectively. X,Y should be much larger than the average size of the speckle. For example, we choose X=Y=111µm for speckle of 2.2µm size as shown in Fig. 2. In Eq. (10), E〈’〉 stands for the operation of ensemble average over independent samples. To obtain an accurate PSD, we ensemble average 150 independent samples. In Eq. (10), uXY (fx,fy) is the Fourier transform of the zero-mean speckle intensity, . is the mean value of the speckle intensity I(x, y) over the spatial range X,Y. By calculating the Fourier transform of the zero-mean speckle intensity instead of the speckle intensity I(x, y), we exclude the δ-function at the DC component of the PSD. As an example, we show PSD of the calculated 2.2µm size speckle in Fig. 3. The δ-function at zero frequency of the PSD is excluded to calculate the sensor MTF. The solid curve is the calculated PSD plot with ensemble averaging over 150 independent samples. The dashed curve is the polynomial fitting of the calculated PSD curve. The x-axis is the spatial frequency in units of the Nyquist frequency of 2.2µm pixel. We use Ny to stand for the Nyquist frequency. Here Ny=227.3cys/mm for a 2.2µm pixel size sensor.
2.2 Avoid aliasing in the sensor MTF measurement
To maintain high signal at the Nyquist frequency of the sensor, the input PSD for the speckle has a cut-off frequency at 2Ny as shown in Fig. 3. The higher cut-off frequency than Ny causes aliasing artifacts in the PSD of the measured speckle pattern on the sensor. From Eq. (3), we see that the aliasing in PSDS(fx,fy) causes inaccurate measurement of sensor MTF. To avoid aliasing artifacts in the measurement, we sub-sample the speckle pattern on the sensor with half pixel size so that the sub-sampled speckle image has Nyquist frequency of 2Ny. The sub-sampling is done by recording the speckle pattern on the sensor at a zero shifting location, then shifting the sensor by a half-pixel along the x-axis location, shifting the sensor by a half-pixel along the y-axis location and shifting the sensor by half-pixels along both the x- and the y-axes location. We then obtain the half-pixel sampled speckle image by combining these four images using the Generalized Sampling Theorem . The PSDS(fx,fy) of the half-pixel sampled speckle image has a Nyquist frequency of 2Ny. There are no aliasing artifacts on it. Figs. 4 show the concept of our anti-aliasing method. Mathematically, the anti-aliasing by sub-sample is achieved by the following procedure:
where Ib is the captured speckle pattern, h is the impulse response function of the image sensor, I is the input speckle pattern on the sensor, P is the pixel size of the sensor and ri, sj are the sub-sample shift movements along the x- and y-axes, respectively. ⊗ stands for the convolution operator. The Fourier transform of Eq. (11) is expressed in Eq. (12) as:
where χb is the Fourier transform of Ib, H is the Fourier transform of h, and χ is the Fourier transform of I. From Eq. (12), we can see that the Fourier transform of the detected laser speckle pattern consists of the non-aliased term for m=0, n=0 and higher order terms when m≠0 or n≠0. To avoid aliasing, we choose ri and sj as r 0=0, r 1=0.5P, s 0=0 and s 1=0.5P for the four sub-sample recording positions. With this choice, the m=0,n=±1 term, m=±1,n=0 term and m=±1,n=±1 term are all zero in Eq. (12). The even higher orders are not considered because the laser speckle pattern has limited bandwidth of 2Ny.
The inaccuracy of the sub-sample shift movements r 1 and s 1 could lead to non-zero higher order terms in Eq. (12) and therefore the error in sensor MTF. Let us define the deviation of r 1 and s 1 from half pixel as Δx and Δy. Δx and Δy are in the range of (-0.5P, 0.5P). We require each aliased term such as 1, m=n=0 be less than five percentages of the nonaliased term to guarantee less than five percentages error in sensor MTF measurement. This requirement could be written as Eq. (13) for m=1,n=0 term.
To obtain five percentages error in the measurement, Δx, Δy≤4*10-3 P. For a 2.2µm pixel-sized sensor, the translational stage for the sub-sample movement needs a resolution of 8.75nm. This number is on the same order of magnitude as the sensor pixel alignment tolerance which is a few nanometers. This requirement on the sub-sample shift movement can be met with commercial ultra-precision motor stages which have 1nm resolution.
3. Experimental results
We present the MTF measurement of Aptina’s monochrome CMOS sensor MI350 with 6.0µm pixel size in Section 3.1. We investigate the applicability of the developed technique to measure MTF of the monochrome image sensor to avoid effects of color filters on sensor MTF measurements. The applicability of the developed methodology to color image sensors will be considered in the future. The speckle method is compared with the slanted edge method. The data from two methods are within excellent agreement. The results for another Aptina’s monochrome CMOS sensor MI5100 with 2.2µm pixel size are described in Section 3.2.
3.1 On-axis MTF measurement for a monochrome CMOS sensor with 6.0µm pixels
Using Eq. (9) and a pixel size of P=6.0µm, we locate the sensor at Z=237mm away from the 25mm OMG diffuser. Image sensor works in the linear mode of operation. Integration time is adjusted to provide the signal from the brightest part of the speckle pattern close to the linear full well capacity. All digital correction circuitries are disabled. The speckle pattern is recorded in a raw image data format. To measure the on-axis sensor MTF, the OMG diffuser and the image sensor are both placed perpendicular to the optical axis which is determined by the collimated He-Ne laser light. To obtain the PSD of the speckle on the sensor, we ensemble average over speckle patterns generated with 25 OMG diffusers. For each OMG diffuser, we record speckle patterns at 4 locations- no sensor shift, sensor shifted by 3.0µm along the x-axis with a linear piezoelectric transducer (PZT), sensor shifted by 3.0µm along the y-axis and sensor shifted by 3.0µm along both the x- and the y-axes. The x-axis is along the row direction of the sensor. The y-axis is along the column direction of the sensor. At each location, multiple frames are recorded and then averaged to eliminate temporal noise of the sensor. As explained in Section 2, we obtain the PSD of the speckle on the sensor, PSDS(fx,fy), from the half pixel sub-sampled speckle patterns on the sensor. The cross-sections of PSDS(fx,fy) along the x- and the y-axes are shown in Figs. 5. The spatial frequency range is from 0 to 2Ny of the sensor. The side lobes in Fig. 5(b) may be due to the limited accuracy of movement along the y-axis of the PZT. The gravity of the PZT stage, sensor and other mechanical mounts may cause the shift of the sensor along the y-axis is not 3µm. This inaccurate shift causes error in the PSD.
The input power spectral density PSDI(fx,fy) is calculated as described in Section 2. The calculated cross-section of PSDI(fx,fy) is shown in Fig. 6 as the dashed black curve. To testify the accuracy of the theoretical calculation of PSDI(fx,fy), we measure the speckle pattern with a 2.2µm pixel size monochrome CMOS sensor. The measured power spectral density PSDs(fx,fy) including the 2.2µm pixel size MTF and the PSD of 6µm-size speckle pattern is plotted in Fig. 6 as the black solid curve. The loss of smoothness in the measured power spectral density curve is due to the limited number of ensemble average. The curve will be smooth if we do ensemble average over more diffusers. To take the 2.2µm pixel-size monochrome CMOS sensor MTF into account, the product of the theoretical calculation of PSDI(fx,fy) and the square of 2.2 µm sensor MTF is plotted as the red dotted curve in Fig. 6. The measurement of 2.2 µm sensor MTF is described in Section 3.2 and the data are shown in Fig. 9. Comparing the three curves in Fig. 6, we can conclude that the theoretical calculation of the input PSD of the speckle on the sensor plane, PSDI(fx,fy), is very accurate.
Using Eq. (3) with calculated input PSDI(fx,fy) and the measured PSDS(fx,fy) data for a 6.0µm CMOS sensor, we obtain the MTF data for the 6.0µm monochrome CMOS sensor. The measured sensor MTF along the x-axis is shown in Fig. 7 as the dotted blue curve. The polynomial fitting of it is shown in Fig. 7 as the solid red curve. As a comparison, the measured MTF along the x-axis using the slanted edge technique is shown in Fig. 7 as the dash-dot green curve. To obtain the MTF data using the slanted edge technique, the ISO12233 target is imaged by a diffraction-limited lens with F# of 9.0 onto the CMOS sensor. A red color filter is used for illumination. The lens F# is chosen to match the equivalent F# of Z/D in the speckle setup. The lens MTF is measured by the Fisba interferometer. The lens and sensor overall MTF is estimated using the Imatest software from the slanted edge image. We obtain the sensor MTF using the slanted edge technique by dividing the overall MTF with the diffraction-limited lens MTF. From Fig. 7, we can see that the laser speckle method and the ISO12233 slanted edge technique provide very similar sensor MTF results. The agreement between the two measurements proves the accuracy of the laser speckle method for MTF measurement of a sensor.
3.2 On-axis MTF measurement for a monochrome CMOS sensor with 2.2µm pixels
With the development of CMOS technology, the pixel size has shrunk dramatically. With small pixel size less than 2µm, the slanted edge technique is limited by the lens quality. Due to the fact that laser speckle method does not require using a lens, we consider this approach as a very promising technique for measuring the MTF of modern CMOS image sensors with small and super small pixels. In this section, we describe the experimental results for measuring MTF of Aptina’s MI5100 monochrome image sensor with 2.2µm pixel size. To obtain the input PSD of the speckle with cut-off frequency at twice the Nyquist frequency of the sensor, the sensor is placed at Z=87mm away from the OMG diffuser. The same procedures for a 6.0µm sensor MTF measurement are repeated. The input power spectral density PSDI(fx,fy) is calculated and shown in Fig. 3. The measured PSDS(fx,fy) along the x-axis is shown in Fig. 8.
Using Eq. (3), we could derive the MTF for the sensor. The cross-section of the MTF along the row direction is shown in Fig. 9. As a comparison, the MTF of the sensor is measured using the slanted edge technique with a lens of F# 3.5. These two techniques provide very close results as shown in Fig. 9. This proves the applicability of measuring small pixel sensors with band-limited speckle pattern.
For high-performance integrated optical and digital imaging systems, it is essential to have the capability for measuring the response of the sub-systems and the overall system. In this paper we describe a new method for measuring the MTF, hence the performance of the CMOS detector. We show quantitatively that this metrology system for the MTF is accurate and presents results in agreement with MTF measurements using the standard ISO 12233 slanted edge technique. Importantly, we assert that this novel speckle method is particularly advantageous for sensors even with pixel size down to 1µm. Moreover, the speckle method can be used to measure the MTF of the sensor alone since it does not require the use of a highquality lens nor the fabrication of special test masks.
To measure sensor MTF at high spatial frequencies around the Nyquist frequency of the sensor, we place the sensor at Z away from the diffuser. Z is defined by Eq. (9). For the speckle, we present how to calculate the input power spectral density PSDI(fx,fy) using the Rayleigh Sommerfeld integral. To testify the accuracy of our calculation, we measure the PSD of 6µm-size speckle by a 2.2µm pixel-size sensor. The theoretical calculation and measurement data are presented in Fig. 6. They agree with each other very well.
Two monochrome CMOS sensors are measured. To avoid aliasing artifacts due to the insufficient sampling of the speckle patterns by the sensors, we sub-sample the speckle patterns by shifting the sensors with half pixel size steps. The anti-aliasing concept is shown in Figs. 4. The error due to inaccurate sub-sample shift movements is discussed. The requirement on sub-sample shift accuracy for five percentages error or less is provided in Eq. (13). The MTF data for a 6.0µm pixel sensor are shown in Fig. 7 compared with the MTF data measured by the ISO12233 slanted edge method. The two results are in excellent agreement. The MTF data for a 2.2µm pixel sensor are shown in Fig. 9. The results obtained with the speckle technique and the slanted edge technique are very close. This agreement proves the accuracy of our laser speckle technique. As a next step we are planning to extend this new MTF measurement technique on color sensors and sensors with shifted µlenses used in mobile applications.
We acknowledge Dr. Wanli Chi at the Institute of Optics, University of Rochester for his helpful discussions and suggestions. We acknowledge our colleagues Doug Fettig, Ian Clark, Sergey Velichko, Feng Li, Dmitry Bakin, Donna Cao, Elaine Jin, Igor Karasev, and Stephen Beveridge for their support.
References and links
1. D. N. Sitter Jr., J. S. Goddard, and R. K. Ferrell, “Method for the measurement of the modulation transfer function of sampled imaging systems from bar-target patterns,” Appl. Opt. 34, 746–751 (1995). [CrossRef] [PubMed]
2. M. Estribeau and P. Magnan, “Fast MTF measurement of CMOS imagers using ISO 12233 slanted edge methodology,” Proc. SPIE 5251, 243–251 (2004). [CrossRef]
3. B. T. Teipen and D. L. MacFarlane, “Liquid-crystal-display projector-based modulation transfer function measurements of charge-coupled-device video camera systems,” Appl. Opt. 39, 515–525 (2000). [CrossRef]
5. M. Sensiper, G. D. Boreman, A. D. Ducharme, and D. R. Snyder, “Modulation transfer function testing of detector arrays using narrow-band laser speckle,” Opt. Eng. 32, 395–400 (1993). [CrossRef]
7. N. George, A. Jain, and R. D. S. Melville Jr., “Speckle, diffusers, and depolarization,” Appl. Phys. 6, 65– 70 (1975). [CrossRef]
8. N. George, A. Jain, and R. D. S. Melville Jr., “Experiments on the space and wavelength dependence of speckle,” Appl. Phys. 7, 157–169 (1975). [CrossRef]
9. N. George, “Speckle at various planes in an optical system,” Opt. Eng. 25, 754–764 (1986).
10. P. Z. Peebles Jr., Probability, random variables, and random signal principles, 3rd Ed. (McGraw-Hill, Inc., New York, 1993).
11. A. Papoulis, “Generalized sampling expansion,” IEEE Trans. Circuits Syst. 24, 652–654 (1977). [CrossRef]