Time delay and integration mode of CCD provides increased sensitivity without the sacrifice of spatial resolution, and the effective integration time is increased by a factor of N
, which is equal to the number of TDI stages [1
U. Bastian and M. Biermann, “Astrometric meaning and interpretation of high-precision time delay integration CCD data,” Astron. Astrophys.
438(2), 745–755 (2005). [CrossRef]
G. Hochman, Y. Yitzhaky, N. S. Kopeika, Y. Lauber, M. Citroen, and A. Stern, “Restoration of images captured by a staggered time delay and integration camera in the presence of mechanical vibrations,” Appl. Opt.
43(22), 4345–4354 (2004). [CrossRef]
], so TDI CCD is widely used in remote sensing systems for improving the low light level capability [3
M. Iyenqar and D. Lange, “The Goodrich 3th generation DB-110 system: operational on tactical and unmanned aircraft,” Proc. SPIE
6209, 1–12 (2006).
A. Lacan, F. M. Bréon, A. Rosak, F. Brachet, L. Roucayrol, P. Etcheto, C. Casteras, and Y. Salaiin, “A static Fourier transform spectrometer for atmospheric sounding: concept and experimental implementation,” Opt. Express
18(8), 8311–8331 (2010). [CrossRef]
]. For example, the most important payload of Chinese lunar spacecraft Chang’eIIis just a TDI CCD stereo camera with 96 stages [5
], its ground sampled distance can reach 1 meter in the 100km × 15km elliptical orbit. As well known, in order to obtain such high resolution, image scan velocity should synchronize with pixel clock rate precisely [6
J. R. Jensen and D. C. Cowen, “Remote sensing of unban/suburban infrastructure and Socio-Economic Attributes,” Photogramm. Eng. Remote Sensing
65(5), 611–622 (1999).
], otherwise image quality degradation will occur [7
J. L. Tong, M. Aydin, and H. E. Bedell, “Direction and extent of perceived motion smear during pursuit eye movement,” Vision Res.
47(7), 1011–1019 (2007). [CrossRef]
]. The camera in Chang’eII is a typical push broom imaging system, thus pixel clock rate is defined as
is detector pitch, and T
is TDI CCD line period. The image scan velocity is expressed as
is aircraft velocity, H
is aircraft altitude. If
, image smear arises, which leads to that the same area on the lunar surface will not be imaged by the same TDI stage. Therefore line scan system is designed such that the image scan velocity should match pixel clock rate as well as possible [8
R. D. Fiete and T. Tantalo, “Image quality of increased along-scan sampling for remote sensing systems,” Opt. Eng.
38(5), 815–820 (1999). [CrossRef]
R. D. Fiete and T. Tantalo, “Comparison of SNR image quality metrics for remote sensing systems,” Opt. Eng.
40(4), 574–585 (2001). [CrossRef]
]. Even so, in most cases resolution in along scan direction is lower than that in cross scan direction. It is because that smear caused by charge transfer of TDI CCD is neglected sometimes. Assuming a TDI CCD adopts Φ
phases to clock out charge in vertical registers. During each TDI stage, charge is clocked out of the detector in discrete steps, meanwhile the imaging system scans the ground continuously. As a result, within the imaging operation of a single TDI stage, Smith [10
S. L. Smith, J. Mooney, T. A. Tantalo, and R. D. Fiete, “Understanding image quality losses due to smear in high resolution remote sensing imaging system,” Opt. Eng.
38(5), 821–826 (1999). [CrossRef]
] and Chamberlain [11
S. G. Chamberlain and W. D. Washkurak, “High-speed, low-noise, fine-resolution TDI CCD imagers,” Proc. SPIE
1242, 1–12 (1990).
] both claimed that 1/Φ
pixel smear would be introduced due to charge transfer in the along scan direction, while Ma [12
T. B. Ma, Y. F. Guo, and Y. F. Li, “Precision of row frequency of scientific grade TDI CCD camera,” Opt. Precis. Eng.
18(9), 2028–2035 (2010).
] reported that 1 pixel smear would occur. Smith and Chamberlain analysis were based on the assumption that interval of charge transfer from one step to another was a fixed value, while Ma didn’t take account of the influence of charge transfer on image smear. Actually, the average charge transfer velocity in a step is likely to vary from one to the other, so the clocking smear model becomes more complicated, and imaging system MTF losses and image quality degradation should be analyzed further [13
G. C. Holst, Electro-optical imaging system performance (SPIE Optical Engineering Press, 2008).
S. A. Cota, C. J. Florio, D. J. Duvall, and M. A. Leon, “The use of general image quality equation in the design and evaluation of imaging systems,” Proc. SPIE
7458, 1–20 (2009).
From the above, we learn that there has been little research on the assessment and validation of TDI CCD clocking smear, and either its formulation is misunderstood, or its influence on image quality is neglected. So there are two main objectives in this paper. First, we will try to develop a more accurate mathematical model of clocking smear for multi phase TDI CCD; and second, to design a new charge transfer method, using which image quality losses due to clocking smear will be reduced to a negligible level.
2. Causes of clocking smear
illustrates a typical TDI CCD structure. It consists of three main parts: imaging rows, isolation row and readout shift registers [15
G. C. Holst, CMOS/CCD sensor and camera system (SPIE Optical Engineering Press, 2007).
]. Both the imaging and isolation rows are controlled by the same three phase timing, while the readout shift registers are controlled by the four phase timing.
Fig. 1 Typical structure of a TDI CCD.
For simplicity, we use the concept of bucket brigade to describe the TDI CCD operation process. As Fig. 2
shows, a number of buckets (pixels) are distributed across a field (focal plane of camera) in a square array. The red and green buckets are placed on top of a series of parallel conveyor belts and collect rain fall (photons) across the field. It is required that the conveyor belts move velocity should synchronize with cloud move rate. In this way, each red bucket will only collect rain fall generated by a fixed piece of cloud until it reaches the end of the conveyor belts, and the green buckets will only receive stored rain from the red buckets without taking the initiative to collect rain from the cloud, since there is a shied on top of it. Finally the collected rain is emptied into blue bucket system on a belt that carries it to a metering station where its contents are measured.
Fig. 2 Bucket analogy for describing TDI CCD operation procedure by traditional phase timing.
Generally, motion of the cloud is continuous, but movement of conveyor belts is discrete determined by charge transfer mode in vertical registers. Figure 3(a)
shows how charge packets are transferred in a three phase TDI CCD, and Fig. 3(b)
illustrates the corresponding phase timing. At time T1
, the potential for phase-1 is held high, forming collecting wells under these gates, charges generated in phase-1 will be collected under phase-1, electrons generated in the barrier phases (phase-2 and phase-3) will rapidly diffuse into phase-1 because its potential is greater, it seems as if there exists a bucket placed at the center of phase-1, and electrons generated in the three phases are all stored in it. At time T2
, the potential for phase-2 is held high, forming the same well under phase-2 that exists under phase-1, so the signal charge packets collected now are divided between the two wells, and charges generated in phase-1 and phase-2 will be collected under potential wells of themselves, since potential of phase-3 is greater, one half of the charges generated in phase-3 will diffuse into its left hand collecting wells, and the other half of charges will diffuse into its right hand collecting wells, it looks as if the bucket moves 1/6 pixel at the beginning of time T2
. At time T3
, phase-1 is returned to ground, which forces charges to transfer to phase-2, the bucket continues to move 1/6 pixel. At time T4
, in the same way, charge packets are transferred from phase-2 to phase-3, and the bucket moves 1/6 pixel once again. This process continues until the charge packets have been moved entirely in 6 charge transfer steps from left to right [16
J. R. Janesick, Scientific charge-coupled devices (SPIE Optical Engineering Press, 2000).
Fig. 3 Block diagram of traditional charge transfer. (a)Charge transfer procedure in a line period, it seems like that one collecting bucket moves 1/6 pixel at the transition moment from one charge transfer step to another; (b) Timing diagram of the three phase TDI CCD.
From above analysis, we know that charge transfer will only take place at the phase timing transition moment, and it could be modeled as
represents detector pitch, ϕ
represents number of timing phases, and
represents charge transfer velocity.
Taking the derivation of Eq. (3)
, the charge transfer velocity is given by
represents duration of charge transfer. The pixel clock rate in one line period could be expressed as the summation of the
individual charge transfer velocities given in Eq. (4)
represents the instant that charge transfer occurs. Compared with the traditional pixel clock rate given in Eq. (1)
, Eq. (5)
provides a more accurate formulation, since the charge transfer procedure of TDI CCD is taken account of.
Finally, similar to our bucket brigade analogy, a line of pixels is transferred into readout shift registers. Before the next parallel line transfer, the readout shift registers should transfer each of its charge packets in sequence to output amplifiers which generates a signal proportional to the amount of charges, once all of the charge packets in the horizontal shift registers have been read out through the output amplifiers, the next line could be transferred [17
G. E. Healey and R. Kondepudy, “Radiometric CCD camera calibration and noise estimation,” IEEE Trans. Pattern Anal. Mach. Intell.
16(3), 267–276 (1994). [CrossRef]
], therefore one of the
charge transfer steps, during which charge packets are reading out in horizontal shift registers, will be longer than the others. From Eq. (2)
and Eq. (5)
, we illustrate the velocity of photosensitive pixel and image point in Fig. 4(a)
, and the relative motion between photosensitive pixel and image point is shown in Fig. 4(b)
. The displacement difference between the red real line (photosensitive pixel) and blue dashed line (image point) is just the cause of clocking smear.
Fig. 4 Photosensitive pixel and image point movement in one line period for a four phase TDI CCD. (a) Velocity comparison between photosensitive pixel and image point. (b) Displacement comparison between photosensitive pixel and image.
3. Mathematical model of clocking smear
In this section, we will present a clocking smear mathematical model caused by charge transfer in the vertical registers. Assuming
is the original image, the image blurred by clocking smear is written as
, and it could be modeled by convolving a rect function with
represents the normalized integration time, which is defined as the ratio of one charge transfer step period to one line period, and l
represents the amount of smear in the along scan direction occurred in one charge transfer step.
When TDI CCD uses ϕ phase timing in vertical registers, there will be charge transfer steps in one line period. According to the description in section 2, the image point will move along the focal plane of the camera during time , where represents time of , and T represents a line period. Meanwhile charge transfer is terminated when image point moves, so the amount of clocking smear occurred at time can be written as
Noting that the normalized integration time at is , we can express the blurred image generated at as
At the beginning of
, the image point has moved
along focal plane, and photosensitive pixel moves
pixel at the transition moment from
, so initial imaging coordinate becomes
, therefore the blurred image generated at
can be written as
is equal to
In this way, we can express the universal blurred image generated in the
charge transfer step as
The final image will be the sum of the individual images collected at the steps, and it can be written as
Taking the Fourier transform yields
Then the MTF of the clocking smear can be expressed as
In practice, the charge readout time in horizontal shift registers accounts for about 70% line period. For simplicity, we assume that
,when the charge readout process occurs, is just equal to 75% line period. According to Eq. (7)
, we summarize pixel smear generated in every charge transfer steps for two, three and four phase TDI CCD in Table.1
Table 1 Pixel Smear Caused by Traditional Phase Timing in Every Charge Transfer Step
|Two phase||Three phase||Four phase|
|| ×|| ×||b/28|
|| ×|| ×||b/28|
Substituting pixel smear listed in Table.1 into Eq. (13)
, we plot a clocking smear MTF diagram in Fig. 5
for the two, three and four phase TDI CCD, where the spatial frequency is normalized to cycles/pixel [18
G. C. Holst, “Imaging system performance based upon Fλ/d,” Opt. Eng.
46(10), 1–8 (2007). [CrossRef]
]. Then we learn that MTF losses due to traditional phase timing is equal to MTF degradation caused by a pixel relative motion between the photosensitive pixel and the image point approximately. Since this kind of smear is the natural component which couldn’t be eliminated by other motion compensation methods, it leads to that the resolution in along scan direction is lower than that in cross scan direction for a line scan system.
Fig. 5 MTF losses due to traditional timing for two, three and four phase TDI CCD, also of MTF losses due to one pixel smear is given for comparison.
4. Clocking smear reduction method
Conventional vertical registers including imaging and isolation rows are controlled by the same phase timing. As a result, some charge transfer step, such as T1 mentioned above, accounts for too much line period, which leads to that relative motion between photosensitive pixel and image point nearly approaches one pixel. In this section we will present a new phase timing scheme, using which the smear caused by TDI CCD charge transfer is reduced to a negligible value.
We still use the bucket brigade model to introduce our method. As Fig. 6
shows, the conveyor belts are divided into two parts, which are driven by two motors respectively. The first conveyor belts (imaging rows) transfer the red buckets one by one in
(number of phases for TDICCD) discrete steps, both the movement distance and the period of each step are equal, so that relative motion between the red bucket and cloud is limited to
, where l
represents distance between two adjacent buckets. In the same way, the second conveyor belts (isolation row) still transfer the green buckets in
discrete steps, and the movement distance of each step is equal. However, the period of all steps are equal except a special one, and it is required that period of the special transfer step accounts for more than 70% line period in order to ensure that there is enough time reserved for blue bucket transfer.
Fig. 6 Bucket analogy for describing TDI CCD operation procedure by our phase timing.
4.1 Imaging rows phase timing
Then, our charge transfer method is presented in detail taking a three phase TDI CCD as the example. As Fig. 7(a)
illustrates, there are 8 charge transfer steps in one line period for the imaging rows, but the last three steps (T6_1, T6_2, T6_3) are completed identical, if we merge the three steps into one step (T6), we will learn that there is no difference between our and traditional charge transfer methods for imaging rows expect period of each charge transfer step. As discussed above, distribution of line transfer period evenly will generate minimum pixel smear, so we divide one line period into 6 intervals equally, in this way, fixed 1/6 pixel smear will be introduced in one line period, and timing of the three phase clocks, CI1, CI2 and CI3 are illustrated in Fig. 7(b)
Fig. 7 Block diagram of our charge transfer method. (a) The process of charge transfer procedure in one line period; (b) Timing diagram of the new charge transfer method.
4.2 Isolation row phase timing
As we know, even division of line transfer period will lead to unwanted electrons diffusion when charge packets are reading out from shift registers, so traditional TDI CCD timing has to adopt non uniform charge transfer intervals to prevent unwanted electrons spread, but the by-product (excessive amount of pixel smear) is also brought. In order to solve the problem, we introduce three new timing signals LCI1, LCI2 and LIC3 to control isolation row solely. As Fig. 7(a)
shows, there are 8 charge transfer steps in one line period for isolation rows, charge transfer from the vertical registers into readout shift registers will only take place in the last three clock cycles(T6_1, T6_2, T6_3), it ensures that the first five charge transfer steps could be used for charge readout from shift registers.
As Fig. 7(b)
illustrates, from Time T1 to T5, the potentials for LCI1, LCI2 and LCI3 are held high simultaneously, forming collecting wells under these gates, then charges generated in imaging area will diffuse into these wells by applying the appropriate voltages of CI1, CI2 and CI3 at sequenced time. Meanwhile TCK is held low level, it prevents charge transfer from isolation row to readout shift registers thanks to that the potential of TCK is greater. From time T6_1 to T6_3, TCK is hold high level, so channel barrier separating isolation row from readout shift registers is eliminated, and LCI1, LCI2, LCI3 are held high level in sequence, then charges stored in isolation rows are transferred to the readout shift registers. Since isolation row is covered with a light shield, it will not lead to additional pixel smear at all. In this way, the proposed phase timing solves the contradiction between timing constraint by charge readout in the shift registers and the amount of pixel smear.
Although our method is analyzed and evaluated based on a three phase TDI CCD, it is also suitable for other phase TDI CCD. From Eq. (6)
, we list the clocking smear caused by our phase timing for two, three and four phase TDI CCD in Table.2.
After substituting pixel smear listed in Table.1 into Eq. (13)
, we illustrate MTF degradation for two, three and four phase TDI CCD in Fig. 8
, then we learn that MTF losses due to our phase timing is equal to MTF degradation caused by 1/4, 1/6 and 1/8 pixel relative motion between photosensitive pixel and image point for two, three and four phase TDI CCD respectively. Compared with Fig. 5
, almost 3/4, 5/6 and 7/8 pixel smears are reduced when adopting our phase timing.
Fig. 8 MTF degradation caused by new phase timing for two, three and four phase TDI CCD.
Table 2 Pixel Smear Caused by New Phase Timing in Every Charge Transfer Step
|Two phase||Three phase||Four phase|
|| ×|| ×||b/8|
|| ×|| ×||b/8|
5. Experimental results
Generally, charge transfer in a full frame CCD will take place after the mechanical shutter is closed. As a result, no clocking smear will be introduced if speed of the shutter is fast enough. In this section, the image quality improvement from our phase timing is assessed by modeling a remote sensing system. Figure 9
illustrates the block diagram of the experimental platform, which mainly consists of bar target pattern, collimator, remote area array camera, mirror, and optics vibration isolation platform. In order to ensure that no additional motion will be introduced between the bar target to be imaged and the imaging system during exposure, all of the equipments are separated from the ground by the stable foundations.
Fig. 9 Structure of the experimental platform.
When the bar target pattern is placed at the collimator’s focus position [19
P. X. Silveira and R. Narayanswamy, “Signal-to-noise analysis of task-based imaging systems with defocus,” Appl. Opt.
45(13), 2924–2934 (2006). [CrossRef]
], the lens converts spatial coordinates at the object plane to image coordinates according to the following proportional relation [20
W. J. Smith, Modern optical engineering (The McGraw-Hill Companies, Inc, 2008).
represents width of a bar,
represents width of the bar image,
represents focus length of the collimator and
represents focus length of the camera. For a fixed focal length, if lower spatial frequencies are required, a larger target must be used, while for a given required spatial frequency, if the target size is constrained, a collimator of shorter focal length is required. In our experiment, the focal length of the collimator and camera are equal to 4000mm and 1000mm respectively, given the range in physical size of our set of bar targets, the resulting spatial frequency range is 24 lines/mm for the largest target to 45 lines/mm for the smallest target according to Eq. (14)
. Noting that Nyquist frequency determined by the CCD pixel pitch (13um) is 38.5lines/mm, we take the middle target, resulting spatial frequency of which is 38 lines/mm, as simulation objects. In this way, MTF improvement by our method could be perceived observably.
Assuming that there exists an analogue imaging system, structures of which are completely identical to the area array camera except that the frame CCD is substituted by a TDI CCD. In this way, image quality degradation will occur due to that the discrete charge transfer procedure results in system MTF losses. Taking account of clocking smear, we illustrate MTF degradation by traditional and our phase timing in Fig. 10
for the modeled TDI CCD camera (The amplitude MTF of the original imaging system at spatial frequency of 0.5cycles/pixel is equal to 0.2).
Fig. 10 The amplitude of MTF by traditional and our phase timing at spatial frequency of 0.5cycles/pixel for the modeled TDI CCD camera.
Then assuming the TDI CCD camera scans the ground in horizontal direction, Fig. 11
illustrates three pairs of image simulations, which are generated by traditional and our phase timing for two-phase, three-phase and four-phase TDICCD respectively. It can be seen that bar targets obtained by our method perform sharper edge and exhibit higher contrast than conventional method in horizontal direction, it is because that MTF degradation caused by our method is much less than the traditional method [21
R. D. Fiete, “Image quality and λFN/p for remote sensing systems,” Opt. Eng.
38(7), 1229–1240 (1999). [CrossRef]
Fig. 11 Image simulations for two, three, and four phase TDI CCD by conventional and proposed phase timing, and charge transfer occurs in horizontal direction (a) conventional two phase timing, (b) conventional three phase timing, (c)conventional four phase timing, (d)proposed two phase timing, (e) proposed three phase timing, (f)proposed four phase timing.
In order to improve performance of remote sensing system, line scan camera designers pay great attention on reducing relative motion between photosensitive pixel and image point. However, the motion caused by charge transfer is neglected sometimes, and its mathematical model is seldom studied. In this paper, we develop an elaborate mathematical model describing multi phase TDI CCD charge transfer procedure, which shows that existing phase timing would lead to nearly one entire pixel smear, therefore the root cause of resolution in along scan direction is lower than that in cross scan direction is disclosed. Then we propose a novel timing method for ϕ phase TDICCD, taking timing signals for the imaging and isolation rows independently, only pixel smear will be introduced during imaging process for one single TDI stage. Compared with traditional phase timing, almost 3/4, 5/6 and 7/8 pixel smears are reduced for two, three and four phase TDI CCD by our timing scheme. Finally the image simulation result favors our analysis well.