Clocking smear caused by charge transfer of time delay and integration charge coupled device (TDI CCD) is the natural component in remote imaging sensing system, and it could not be eliminated by traditional motion compensation schemes. After researching on the operation of a typical three phase TDI CCD, we give a thorough understanding on causes of clocking smear. Then an elaborate mathematical model describing the charge transfer procedure is developed, and the modulation transfer function (MTF) losses due to charge transfer is also presented, which shows that nearly one pixel smear will be introduced by traditional phase timing. Therefore we propose a novel charge transfer method, using which only pixel smear will occur within the imaging operation of a single TDI stage, where ϕ represents the number of timing phases. Finally, a series of image simulations are made for two, three and four phase TDI CCD in which clocking smear is caused by our and conventional charge transfer methods respectively. The experimental results confirm that image quality improvement can be achieved by our method.
©2011 Optical Society of America
Time delay and integration mode of CCD provides increased sensitivity without the sacrifice of spatial resolution, and the effective integration time is increased by a factor of N, which is equal to the number of TDI stages [1,2], so TDI CCD is widely used in remote sensing systems for improving the low light level capability [3,4]. For example, the most important payload of Chinese lunar spacecraft Chang’eIIis just a TDI CCD stereo camera with 96 stages , its ground sampled distance can reach 1 meter in the 100km × 15km elliptical orbit. As well known, in order to obtain such high resolution, image scan velocity should synchronize with pixel clock rate precisely , otherwise image quality degradation will occur . The camera in Chang’eII is a typical push broom imaging system, thus pixel clock rate is defined as8,9]. Even so, in most cases resolution in along scan direction is lower than that in cross scan direction. It is because that smear caused by charge transfer of TDI CCD is neglected sometimes. Assuming a TDI CCD adopts Φ phases to clock out charge in vertical registers. During each TDI stage, charge is clocked out of the detector in discrete steps, meanwhile the imaging system scans the ground continuously. As a result, within the imaging operation of a single TDI stage, Smith  and Chamberlain  both claimed that 1/Φ pixel smear would be introduced due to charge transfer in the along scan direction, while Ma  reported that 1 pixel smear would occur. Smith and Chamberlain analysis were based on the assumption that interval of charge transfer from one step to another was a fixed value, while Ma didn’t take account of the influence of charge transfer on image smear. Actually, the average charge transfer velocity in a step is likely to vary from one to the other, so the clocking smear model becomes more complicated, and imaging system MTF losses and image quality degradation should be analyzed further [13,14].
From the above, we learn that there has been little research on the assessment and validation of TDI CCD clocking smear, and either its formulation is misunderstood, or its influence on image quality is neglected. So there are two main objectives in this paper. First, we will try to develop a more accurate mathematical model of clocking smear for multi phase TDI CCD; and second, to design a new charge transfer method, using which image quality losses due to clocking smear will be reduced to a negligible level.
2. Causes of clocking smear
Figure 1 illustrates a typical TDI CCD structure. It consists of three main parts: imaging rows, isolation row and readout shift registers . Both the imaging and isolation rows are controlled by the same three phase timing, while the readout shift registers are controlled by the four phase timing.
For simplicity, we use the concept of bucket brigade to describe the TDI CCD operation process. As Fig. 2 shows, a number of buckets (pixels) are distributed across a field (focal plane of camera) in a square array. The red and green buckets are placed on top of a series of parallel conveyor belts and collect rain fall (photons) across the field. It is required that the conveyor belts move velocity should synchronize with cloud move rate. In this way, each red bucket will only collect rain fall generated by a fixed piece of cloud until it reaches the end of the conveyor belts, and the green buckets will only receive stored rain from the red buckets without taking the initiative to collect rain from the cloud, since there is a shied on top of it. Finally the collected rain is emptied into blue bucket system on a belt that carries it to a metering station where its contents are measured.
Generally, motion of the cloud is continuous, but movement of conveyor belts is discrete determined by charge transfer mode in vertical registers. Figure 3(a) shows how charge packets are transferred in a three phase TDI CCD, and Fig. 3(b) illustrates the corresponding phase timing. At time T1, the potential for phase-1 is held high, forming collecting wells under these gates, charges generated in phase-1 will be collected under phase-1, electrons generated in the barrier phases (phase-2 and phase-3) will rapidly diffuse into phase-1 because its potential is greater, it seems as if there exists a bucket placed at the center of phase-1, and electrons generated in the three phases are all stored in it. At time T2, the potential for phase-2 is held high, forming the same well under phase-2 that exists under phase-1, so the signal charge packets collected now are divided between the two wells, and charges generated in phase-1 and phase-2 will be collected under potential wells of themselves, since potential of phase-3 is greater, one half of the charges generated in phase-3 will diffuse into its left hand collecting wells, and the other half of charges will diffuse into its right hand collecting wells, it looks as if the bucket moves 1/6 pixel at the beginning of time T2. At time T3, phase-1 is returned to ground, which forces charges to transfer to phase-2, the bucket continues to move 1/6 pixel. At time T4, in the same way, charge packets are transferred from phase-2 to phase-3, and the bucket moves 1/6 pixel once again. This process continues until the charge packets have been moved entirely in 6 charge transfer steps from left to right .
From above analysis, we know that charge transfer will only take place at the phase timing transition moment, and it could be modeled as
Taking the derivation of Eq. (3), the charge transfer velocity is given byEq. (4) Eq. (1), Eq. (5) provides a more accurate formulation, since the charge transfer procedure of TDI CCD is taken account of.
Finally, similar to our bucket brigade analogy, a line of pixels is transferred into readout shift registers. Before the next parallel line transfer, the readout shift registers should transfer each of its charge packets in sequence to output amplifiers which generates a signal proportional to the amount of charges, once all of the charge packets in the horizontal shift registers have been read out through the output amplifiers, the next line could be transferred , therefore one of the charge transfer steps, during which charge packets are reading out in horizontal shift registers, will be longer than the others. From Eq. (2) and Eq. (5), we illustrate the velocity of photosensitive pixel and image point in Fig. 4(a) , and the relative motion between photosensitive pixel and image point is shown in Fig. 4(b). The displacement difference between the red real line (photosensitive pixel) and blue dashed line (image point) is just the cause of clocking smear.
3. Mathematical model of clocking smear
In this section, we will present a clocking smear mathematical model caused by charge transfer in the vertical registers. Assuming is the original image, the image blurred by clocking smear is written as , and it could be modeled by convolving a rect function with
When TDI CCD uses ϕ phase timing in vertical registers, there will be charge transfer steps in one line period. According to the description in section 2, the image point will move along the focal plane of the camera during time , where represents time of , and T represents a line period. Meanwhile charge transfer is terminated when image point moves, so the amount of clocking smear occurred at time can be written as
Noting that the normalized integration time at is , we can express the blurred image generated at as
At the beginning of , the image point has moved along focal plane, and photosensitive pixel moves pixel at the transition moment from to , so initial imaging coordinate becomes , therefore the blurred image generated at can be written as
In this way, we can express the universal blurred image generated in the charge transfer step as
The final image will be the sum of the individual images collected at the steps, and it can be written as
Taking the Fourier transform yields
Then the MTF of the clocking smear can be expressed as
In practice, the charge readout time in horizontal shift registers accounts for about 70% line period. For simplicity, we assume that ,when the charge readout process occurs, is just equal to 75% line period. According to Eq. (7), we summarize pixel smear generated in every charge transfer steps for two, three and four phase TDI CCD in Table.1
Substituting pixel smear listed in Table.1 into Eq. (13), we plot a clocking smear MTF diagram in Fig. 5 for the two, three and four phase TDI CCD, where the spatial frequency is normalized to cycles/pixel . Then we learn that MTF losses due to traditional phase timing is equal to MTF degradation caused by a pixel relative motion between the photosensitive pixel and the image point approximately. Since this kind of smear is the natural component which couldn’t be eliminated by other motion compensation methods, it leads to that the resolution in along scan direction is lower than that in cross scan direction for a line scan system.
4. Clocking smear reduction method
Conventional vertical registers including imaging and isolation rows are controlled by the same phase timing. As a result, some charge transfer step, such as T1 mentioned above, accounts for too much line period, which leads to that relative motion between photosensitive pixel and image point nearly approaches one pixel. In this section we will present a new phase timing scheme, using which the smear caused by TDI CCD charge transfer is reduced to a negligible value.
We still use the bucket brigade model to introduce our method. As Fig. 6 shows, the conveyor belts are divided into two parts, which are driven by two motors respectively. The first conveyor belts (imaging rows) transfer the red buckets one by one in (number of phases for TDICCD) discrete steps, both the movement distance and the period of each step are equal, so that relative motion between the red bucket and cloud is limited to , where l represents distance between two adjacent buckets. In the same way, the second conveyor belts (isolation row) still transfer the green buckets in discrete steps, and the movement distance of each step is equal. However, the period of all steps are equal except a special one, and it is required that period of the special transfer step accounts for more than 70% line period in order to ensure that there is enough time reserved for blue bucket transfer.
4.1 Imaging rows phase timing
Then, our charge transfer method is presented in detail taking a three phase TDI CCD as the example. As Fig. 7(a) illustrates, there are 8 charge transfer steps in one line period for the imaging rows, but the last three steps (T6_1, T6_2, T6_3) are completed identical, if we merge the three steps into one step (T6), we will learn that there is no difference between our and traditional charge transfer methods for imaging rows expect period of each charge transfer step. As discussed above, distribution of line transfer period evenly will generate minimum pixel smear, so we divide one line period into 6 intervals equally, in this way, fixed 1/6 pixel smear will be introduced in one line period, and timing of the three phase clocks, CI1, CI2 and CI3 are illustrated in Fig. 7(b).
4.2 Isolation row phase timing
As we know, even division of line transfer period will lead to unwanted electrons diffusion when charge packets are reading out from shift registers, so traditional TDI CCD timing has to adopt non uniform charge transfer intervals to prevent unwanted electrons spread, but the by-product (excessive amount of pixel smear) is also brought. In order to solve the problem, we introduce three new timing signals LCI1, LCI2 and LIC3 to control isolation row solely. As Fig. 7(a) shows, there are 8 charge transfer steps in one line period for isolation rows, charge transfer from the vertical registers into readout shift registers will only take place in the last three clock cycles(T6_1, T6_2, T6_3), it ensures that the first five charge transfer steps could be used for charge readout from shift registers.
As Fig. 7(b) illustrates, from Time T1 to T5, the potentials for LCI1, LCI2 and LCI3 are held high simultaneously, forming collecting wells under these gates, then charges generated in imaging area will diffuse into these wells by applying the appropriate voltages of CI1, CI2 and CI3 at sequenced time. Meanwhile TCK is held low level, it prevents charge transfer from isolation row to readout shift registers thanks to that the potential of TCK is greater. From time T6_1 to T6_3, TCK is hold high level, so channel barrier separating isolation row from readout shift registers is eliminated, and LCI1, LCI2, LCI3 are held high level in sequence, then charges stored in isolation rows are transferred to the readout shift registers. Since isolation row is covered with a light shield, it will not lead to additional pixel smear at all. In this way, the proposed phase timing solves the contradiction between timing constraint by charge readout in the shift registers and the amount of pixel smear.
Although our method is analyzed and evaluated based on a three phase TDI CCD, it is also suitable for other phase TDI CCD. From Eq. (6), we list the clocking smear caused by our phase timing for two, three and four phase TDI CCD in Table.2.
After substituting pixel smear listed in Table.1 into Eq. (13), we illustrate MTF degradation for two, three and four phase TDI CCD in Fig. 8 , then we learn that MTF losses due to our phase timing is equal to MTF degradation caused by 1/4, 1/6 and 1/8 pixel relative motion between photosensitive pixel and image point for two, three and four phase TDI CCD respectively. Compared with Fig. 5, almost 3/4, 5/6 and 7/8 pixel smears are reduced when adopting our phase timing.
5. Experimental results
Generally, charge transfer in a full frame CCD will take place after the mechanical shutter is closed. As a result, no clocking smear will be introduced if speed of the shutter is fast enough. In this section, the image quality improvement from our phase timing is assessed by modeling a remote sensing system. Figure 9 illustrates the block diagram of the experimental platform, which mainly consists of bar target pattern, collimator, remote area array camera, mirror, and optics vibration isolation platform. In order to ensure that no additional motion will be introduced between the bar target to be imaged and the imaging system during exposure, all of the equipments are separated from the ground by the stable foundations.
When the bar target pattern is placed at the collimator’s focus position , the lens converts spatial coordinates at the object plane to image coordinates according to the following proportional relation Eq. (14). Noting that Nyquist frequency determined by the CCD pixel pitch (13um) is 38.5lines/mm, we take the middle target, resulting spatial frequency of which is 38 lines/mm, as simulation objects. In this way, MTF improvement by our method could be perceived observably.
Assuming that there exists an analogue imaging system, structures of which are completely identical to the area array camera except that the frame CCD is substituted by a TDI CCD. In this way, image quality degradation will occur due to that the discrete charge transfer procedure results in system MTF losses. Taking account of clocking smear, we illustrate MTF degradation by traditional and our phase timing in Fig. 10 for the modeled TDI CCD camera (The amplitude MTF of the original imaging system at spatial frequency of 0.5cycles/pixel is equal to 0.2).
Then assuming the TDI CCD camera scans the ground in horizontal direction, Fig. 11 illustrates three pairs of image simulations, which are generated by traditional and our phase timing for two-phase, three-phase and four-phase TDICCD respectively. It can be seen that bar targets obtained by our method perform sharper edge and exhibit higher contrast than conventional method in horizontal direction, it is because that MTF degradation caused by our method is much less than the traditional method .
In order to improve performance of remote sensing system, line scan camera designers pay great attention on reducing relative motion between photosensitive pixel and image point. However, the motion caused by charge transfer is neglected sometimes, and its mathematical model is seldom studied. In this paper, we develop an elaborate mathematical model describing multi phase TDI CCD charge transfer procedure, which shows that existing phase timing would lead to nearly one entire pixel smear, therefore the root cause of resolution in along scan direction is lower than that in cross scan direction is disclosed. Then we propose a novel timing method for ϕ phase TDICCD, taking timing signals for the imaging and isolation rows independently, only pixel smear will be introduced during imaging process for one single TDI stage. Compared with traditional phase timing, almost 3/4, 5/6 and 7/8 pixel smears are reduced for two, three and four phase TDI CCD by our timing scheme. Finally the image simulation result favors our analysis well.
References and links
1. U. Bastian and M. Biermann, “Astrometric meaning and interpretation of high-precision time delay integration CCD data,” Astron. Astrophys. 438(2), 745–755 (2005). [CrossRef]
2. G. Hochman, Y. Yitzhaky, N. S. Kopeika, Y. Lauber, M. Citroen, and A. Stern, “Restoration of images captured by a staggered time delay and integration camera in the presence of mechanical vibrations,” Appl. Opt. 43(22), 4345–4354 (2004). [CrossRef] [PubMed]
3. M. Iyenqar and D. Lange, “The Goodrich 3th generation DB-110 system: operational on tactical and unmanned aircraft,” Proc. SPIE 6209, 1–12 (2006).
4. A. Lacan, F. M. Bréon, A. Rosak, F. Brachet, L. Roucayrol, P. Etcheto, C. Casteras, and Y. Salaiin, “A static Fourier transform spectrometer for atmospheric sounding: concept and experimental implementation,” Opt. Express 18(8), 8311–8331 (2010). [CrossRef] [PubMed]
5. X. Zhang, “China’s 2nd lunar probe Chang’e-2 blasts off,” (English.xinhuanet.com, 2010) http://news.xinhuanet.com/english2010/sci/2010-10/01/c_13539035.
6. J. R. Jensen and D. C. Cowen, “Remote sensing of unban/suburban infrastructure and Socio-Economic Attributes,” Photogramm. Eng. Remote Sensing 65(5), 611–622 (1999).
8. R. D. Fiete and T. Tantalo, “Image quality of increased along-scan sampling for remote sensing systems,” Opt. Eng. 38(5), 815–820 (1999). [CrossRef]
9. R. D. Fiete and T. Tantalo, “Comparison of SNR image quality metrics for remote sensing systems,” Opt. Eng. 40(4), 574–585 (2001). [CrossRef]
10. S. L. Smith, J. Mooney, T. A. Tantalo, and R. D. Fiete, “Understanding image quality losses due to smear in high resolution remote sensing imaging system,” Opt. Eng. 38(5), 821–826 (1999). [CrossRef]
11. S. G. Chamberlain and W. D. Washkurak, “High-speed, low-noise, fine-resolution TDI CCD imagers,” Proc. SPIE 1242, 1–12 (1990).
12. T. B. Ma, Y. F. Guo, and Y. F. Li, “Precision of row frequency of scientific grade TDI CCD camera,” Opt. Precis. Eng. 18(9), 2028–2035 (2010).
13. G. C. Holst, Electro-optical imaging system performance (SPIE Optical Engineering Press, 2008).
14. S. A. Cota, C. J. Florio, D. J. Duvall, and M. A. Leon, “The use of general image quality equation in the design and evaluation of imaging systems,” Proc. SPIE 7458, 1–20 (2009).
15. G. C. Holst, CMOS/CCD sensor and camera system (SPIE Optical Engineering Press, 2007).
16. J. R. Janesick, Scientific charge-coupled devices (SPIE Optical Engineering Press, 2000).
17. G. E. Healey and R. Kondepudy, “Radiometric CCD camera calibration and noise estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 16(3), 267–276 (1994). [CrossRef]
18. G. C. Holst, “Imaging system performance based upon Fλ/d,” Opt. Eng. 46(10), 1–8 (2007). [CrossRef]
20. W. J. Smith, Modern optical engineering (The McGraw-Hill Companies, Inc, 2008).
21. R. D. Fiete, “Image quality and λFN/p for remote sensing systems,” Opt. Eng. 38(7), 1229–1240 (1999). [CrossRef]