To increase the field of view (FOV), combining multiple time-delayed and integrated charge-coupled devices (TDI-CCD) into the camera and the pushbroom imaging modality are traditionally used with high-resolution optical satellites. It is becoming increasingly labor- and cost-intensive to build and maintain a calibration field with high resolution and broad coverage. This paper introduces a simple and feasible on-orbit geometric self-calibration approach for high-resolution multi-TDI-CCD optical satellites based on three-view stereoscopic images. With the aid of the a priori geometric constraint of tie points in the triple-overlap regions of stereoscopic images, as well as tie points between adjacent single TDI-CCD images (STIs), high accuracy calibration of all TDI-CCD detectors can be achieved using a small number of absolute ground control points (GCPs) covering the selected primary STI. This method greatly reduces the demand on the calibration field and thus is more time-, effort- and cost-effective. Experimental results indicated that the proposed self-calibration approach is effective for increasing the relative internal accuracy without the limitations associated with using a traditional reference calibration field, which could have great significance for future super-high-resolution optical satellites.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
With the increase in demand for high resolution and large field of view (FOV), the multi-linear TDI-CCD array pushbroom imaging modality is widely used with high-resolution optical satellites. Because it is difficult to meet the requirement of a large FOV while guaranteeing high resolution using a single additional time-delayed and integrated charge-coupled device (TDI-CCD) detector, it is necessary to combine multiple TDI-CCD detectors into the camera, as is the case for IKONOS, Pleiades-HR, the Worldview series, and ZiYuan3 . On-orbit geometric calibration is necessary to guarantee the geometric quality of high-resolution optical remote sensing satellite imagery . Rigorous laboratory calibration of optical satellite cameras is always undertaken prior to launch, for example with respect to geometric distortion and the difference in installation angles between the optical camera and star trackers [3, 4]. However, these preset parameters may be altered by variation in the thermal environment and physical stress release. Therefore, the camera must be recalibrated during the mission.
Many investigations have been conducted into on-orbit calibration of optical satellites. Traditional calibration methods are usually based on a high-precision calibration field with digital orthophoto maps (DOMs) and corresponding digital elevation models (DEMs) [5–7]. Ground control points (GCPs) can be generated by matching the satellite images with calibration field images to estimate the external and internal camera parameters. External and internal calibration are commonly included in traditional calibration methods, with external calibration referring to determination of the installation angle of the camera, and internal calibration referring to determination of the camera’s internal distortion. Traditional calibration methods, which are based entirely on the calibration field, can guarantee the geometric accuracy of remote sensing images. However, maintaining a calibration field with high resolution and broad coverage will become increasingly expensive and labor-intensive because of seasonal variation and surface texture changes. With the increasing demand for higher geometric resolution and better FOV of satellites, these drawbacks of on-orbit geometric calibration will become more serious. Therefore, researchers have sought to explore self-calibration methods without the limitations of the calibration field.
Camera self-calibration is widely used for close-range photogrammetry and aerial photogrammetry [8–12]. Faugeras et al. conducted an analysis of the theoretical feasibility and practicality of detecting distortion errors using multi-view images . Malis et al. proposed a self-calibration method using multi-view constraints based on unknown planar structures . However, in contrast to most area array cameras used in close-range photogrammetry and aerial photogrammetry, these methods are not very effective for the linear array pushbroom imaging modality used by satellites. Therefore, in spaceborne photogrammetry, self-calibration by block adjustment is the most common calibration method, in which camera distortion parameters serve as additional parameters in the adjustment calculation [15,16]. Although block adjustment self-calibration has been applied successfully to Chang’E-2, CBERS-02B, and ZiYuan-3 satellite images to improve their geometric accuracy, the long period of data acquisition, complex calculations, and low stability of additional calibration parameters remain significant limitations. Based on the imaging capability of agile optical satellites, pushbroom and swing-scanning images of the same area can be obtained for the self-calibration method proposed by Pleaides . However, this method requires high satellite maneuverability, and thus is not suitable for most satellites. Zhang et al. proposed a correction method of pushbroom satellite imagery interior distortions independent of GCPs . However, they only talked about the correction method of the wide field of view camera on GaoFen1 with single linear CCD and middle ground resolution, which is difficult to apply directly to high resolution satellite with multi-TDI-CCD detectors. Rapid stereoscopic imaging on the same track using forward and backward imaging, and on different tracks using side imaging, are the most common methods used with high-resolution optical satellites at present. Therefore, taking advantage of the a priori geometric constraints in multi-view stereoscopic images is a meaningful avenue of exploration for future research into self-calibration methods.
This paper presents an on-orbit geometric self-calibration approach for high-resolution linear array pushbroom optical remote sensing satellites with multiple TDI-CCD detectors, in which three-view stereoscopic images are used to determine the geometric constraints for self-calibration. First, we designate the intermediate TDI-CCD detector as the primary detector, and its internal distortion parameters can be accurately determined using a small number of GCPs. Based on the same internal distortion parameters of the primary single TDI-CCD images (STIs), the relative external coordinate systems of the three stereoscopic images can be accurately determined. Then, the internal distortion parameters of the other TDI-CCD detectors (hereafter, secondary TDI-CCD detectors) can undertake self-calibration based on accurate determination of the relative attitude of the three images and the intersection constraints of the tie points. Both the shared tie points among the three stereoscopic images and those between adjacent STIs are used for highly accurate estimation of the internal parameters of the secondary TDI-CCD detectors. Two key innovations are presented in this paper:
- 1) In contrast to traditional calibration methods, which are based entirely on the calibration field and generally demand that it has large coverage and is longer than the width of the satellite image, the proposed self-calibration approach uses a small range of absolute GCPs, covering only the primary STI, to achieve high-accuracy calibration of the whole image. This method greatly reduces the calibration field demand and thus is more time-, effort- and cost-effective.
- 2) Based on the tie points shared among the three images and the tie points shared between adjacent STIs, a priori geometric constraints are efficiently used for highly accurate estimation of the internal parameters of the secondary TDI-CCD detectors. In this way, both the internal accuracy of the STI and the mosaicking accuracy between adjacent STIs can be guaranteed. There is no loss of accuracy when stitching STIs together to generate the whole (stitched) images using the relative geometric constraint of each STI.
Experimental results verified that the internal parameters calibrated using the proposed self-calibration approach were effective at increasing the relative internal accuracy. In this way, we can achieve highly accurate geometric calibration of high-resolution and large FOV multi-linear array optical satellites without the limitations associated with the traditional reference calibration field. This study has great significance for future super-high-resolution optical satellites.
2. Imaging model for the multi-TDI-CCD
A rigorous geometric imaging model serves as the foundation for the proposed on-orbit calibration model. The imaging parameters of the rigorous geometric imaging model can be divided into auxiliary satellite data and camera parameters. Auxiliary satellite data include time, attitude and orbit, which help convert satellite body coordinates into object coordinates and thus determine the line of sight (LOS) from the projection center to the ground points in the satellite body coordinate system . The internal camera parameters help to convert the image coordinates into camera coordinates, which can be used to determine the most accurate LOS for each detector within the camera coordinate system. The exterior camera parameters (installation angles) determine the rotational relationship between the satellite body coordinate system and the camera coordinate system.
For a pushbroom camera with multiple TDI-CCD detectors, a rigorous imaging model for each TDI-CCD detector can be created as follows:
where and are the viewing angles between the LOS and the vertical and horizontal axes of the focal plane, respectively. is the sequence number of the single TDI-CCD detector, and and are the corresponding internal parameters. is the object space of the WGS84 ground point coordinate. is the position vector of the projection center within the WGS84 coordinate system, which is interpolated from ephemeris time observations. is the installation matrix between the satellite body coordinate system and the camera coordinate system, while is the attitude matrix between the J2000 coordinate system and the satellite body coordinate system interpolated from the attitude time observations obtained under the J2000 coordinate system. Finally, represents the transformation matrix from the WGS84 coordinate system to the J2000 coordinate system, which is based on the IAU 2000 precession-nutation model according to International Earth Rotation and Reference Systems Service (IERS) conventions.
Traditional calibration parameters include the external installation matrix and the internal directional angle models and , where roll, pitch and yaw are the corresponding installation angles. Because of the high correlation between external and internal calibration parameters caused by unusual imaging conditions, such as a high orbit and narrow field angle, traditional calibration approaches usually adopt a stepwise strategy, in which external parameters are estimated first and then internal parameters are estimated in a generalized frame determined from the external parameters [3, 7].
3. Geometric self-calibration approach
Traditional calibration methods, being based entirely on the calibration field, require labor-intensive and costly maintenance of the calibration field. To reduce dependence on absolute ground reference points, and to define the a priori geometric constraint for self-calibration, we chose three stereoscopic images with an appropriate overlapping region. Because the focal plane of the camera is usually composed of several TDI-CCD detectors within the FOV, to build intersection models for the TDI-CCD detectors each STI within one whole image should have an appropriate triple-overlap region with the other two corresponding STIs, as shown in Fig. 1(a). Three overlapping whole images can be obtained through stereoscopic imaging on the same track by forward and backward imaging, or on different tracks by side imaging.
There are three steps in the self-calibration procedure and the basic process is shown in Fig. 2.
First, we select one TDI-CCD detector as the primary detector, and calibrate the installation angles for image 2 and the internal distortion parameters of the primary TDI-CCD using the traditional stepwise calibration method [3,7]. Image 2 represents the middle of the three stereoscopic images.
Then, the installation angles of images 1 and 3 can be calculated based on the geometric constraint of the tie points in the primary STI pair or the discrete GCPs used in the previous step.
Lastly, evenly distributed tie points in each secondary STI pair are matched so that an intersection model can be built. Based on this model, the internal distortion of the secondary TDI-CCD detectors can be determined and offset.
Using this procedure, only the selected primary TDI-CCD detector requires absolute reference GCPs for calibration, while the secondary detectors can carry out self-calibration based on the relative intersection constraint, which will greatly reduce the reference calibration field requirement.
3.1 Calibration of the primary TDI-CCD detector using GCPs
In general, we utilize the middle TDI-CCD as the primary detector. To calibrate the primary TDI-CCD detector, several discrete GCPs that are evenly distributed in a straight line across the primary TDI-CCD of image 2 should be measured to reduce the negative effects of satellite vibration, as shown in Fig. 1(b) . In engineering applications, the GCPs can be accurately measured using global positioning system (GPS) or a high-accuracy matching method based on the available DOM and DEM. A relatively small number of accurately measured GCPs are required for effective fitting, identification and compensation for the single TDI-CCD distortion curve, and a greater number of GCPs does not present obvious advantages. Measuring a few discrete GCPs using GPS for calibration is a relatively efficient method in terms of time, money, and labor compared with building and maintaining a large-area calibration field with high accuracy and resolution. Traditional stepwise calibration is performed based on GCPs, the installation angles of image 2 are estimated based on the initial internal parameters of the primary TDI-CCD, and the internal parameters are then updated [4,7].
To determine external calibration parameters of image 2, we assume that the initial internal calibration parameters are “true”. We can initialize the external and internal calibration parameters with the initial on-ground calibration results before launch and . is the serial number of GCPs.
According to Eq. (1) we can set:
is the correction of the installation angles. is the residual error vector. Then, the installation angles can be estimated based on iteration least square method .
is the LOS vector of the GCP in the camera coordinate system determined by the modified . Then, the internal parameters can be estimated based on least square method .
In the analysis presented in Section 3.3, we note that precise specification of the relative external reference coordinate system of the three stereoscopic images is necessary for self-calibration based on tie points without an absolute control. The reference coordinate system includes the position and attitude of the satellite and the installation angles of the camera. Based on strong correlations in terms of geometric positioning, the installation angles can compensate well for attitude and position errors . Therefore, determining the installation angles of the three stereoscopic images will in turn allow determination of their reference coordinate systems. Through this method, the installation angles of image 2 and the internal parameters of the primary TDI-CCD detector can be obtained. Due to the strong correlation between the installation and internal directional angles, these values may not reflect the actual installation angles and internal distortions. However, based on a rigorous geometric imaging model, both the LOS in the body coordinate system and the positioning accuracy can be guaranteed using concordant installation angles and internal parameters.
3.2 Determination of the installation angles of images 1 and 3
In the previous step, the installation angles of image 2 and the internal parameters of the primary TDI-CCD detector were determined. To determine the installation angles of images 1 and 3, the GCPs in the previous step can still be utilized as an absolute control. However, if a high-accuracy reference DEM is available, relative geometric controls based on the tie points between the primary STI pair of images 2 and 1, and of images 2 and 3, is a better choice. To reduce the negative effects of attitude error, the tie points should be distributed within a narrow strip around the GCPs used previously, as shown in Fig. 1(c). The WGS84 object space coordinates of the tie points can be obtained by forward-projection based on a rigorous imaging model that uses the estimated installation angles of image 1 and the internal parameters of the primary TDI-CCD. In this way, the image space coordinates of tie points in primary TDI-CCD individual images 1 and 3 and their WGS84 object space coordinates define the mapping relationship, as shown in Fig. 3.
Based on the intersecting ground points , we can insert the estimated primary TDI-CCD internal parameters in Step 1 into Eq. (1) and use Eq. (3)-Eq. (5) to calibrate the installation angles of images 1 and 3. To guarantee the accuracy of the ground points and reduce the demand for accuracy and resolution in the reference DEM, the tie points should be located within a flat region, and tall buildings in a city should be avoided. Although a strong correlation between the installation angles and internal parameters will result in only a minor error in the absolute accuracy of the calculated external reference coordinate system, accurately calibrated primary TDI-CCD internal parameters guarantee the accuracy of external relative reference coordinate systems, reflecting their spatial relationship and meeting the self-calibration requirement described in the next section.
3.3 Self-calibration of the secondary TDI-CCD detectors using tie points
The main objective of the previous two steps was to provide an accurate relative external reference coordinate system for the self-calibration of secondary TDI-CCD detectors. To prevent high-frequency attitude errors from affecting the accuracy of the relative external reference coordinate system determined by the tie points of the primary STI, tie points among three corresponding secondary STIs should also be distributed within a narrow strip to maintain consistency in imaging time with the primary STI, as shown in Fig. 1(d).
3.3.1 Geometric self-calibration model
The general principle of self-calibration is that internal distortion can be detected by three intersecting LOSs via repeated three-view imaging, with the aid of the calibrated accurate relative external attitudes. Then a forward intersection model of the tie points is constructed based on the rigorous imaging model, a schematic diagram of which is shown in Fig. 4.
As shown in Fig. 4, when there is no internal distortion of the camera, object point is forward-intersected by the ideal image tie points , and . However, internal distortion is inevitable, and the ideal image points , and projected from object point are actually imaged at points , and . Because the distortion varies among different parts of the same TDI-CCD detector, the intersection points , and probably differ if the internal distortion is unknown. This difference in the intersection points is called intersection residual error in object space, and causes the LOS residual error in image space.
The intersection or LOS residual errors can be used to detect the internal distortion of the TDI-CCD detector. In consideration of the strong correlation between the external attitude and internal parameters, relative error in the external attitude of the three stereoscopic images can be absorbed into the calibrated internal parameters and reduces the accuracy of calibration. Then, it is impossible to judge whether the relative external attitude error or internal distortion is the cause of the intersection error if the relative accuracy of the external attitude cannot be guaranteed. Therefore, it is worth noting that a precondition for the newly proposed self-calibration approach is an accurate relative external attitude measurement.
Then, based on Eq. (1), the error equation of three stereoscopic images for each forward-intersected object point can be constructed as:
where relates to the right side of Eq. (1). Therefore, the unknown parameters include internal distortion parameters as well as the ground geodetic coordinates of the tie points. Then, we can linearize Eq. (8) for the kth iteration as:
where and is the correction of the internal distortion parameters, , is the number of tie points, and is the correction vector of the ground coordinates. is the residual error vector of the tie points, and and are design matrices consisting of the partial derivatives of calibration parameters calculated using tie point measurements and the ground coordinates of the tie points, respectively. is the LOS difference vector of the tie points in image space, calculated by reference to the internal distortion parameters, and is the weighted matrix of tie points.
3.3.2 Stepwise estimation method for internal parameters of the secondary TDI-CCD
Based on the multi-LOS intersection principle presented above, systematic translation of the TDI-CCD detector is difficult to detect and compensate for using the proposed self-calibration model.
As shown in Fig. 5(a), systematic translation of the TDI-CCD detector does not cause intersection residual error. In Eq. (2), determines translation along the track, while determines translation across the track. Therefore, based on the proposed self-calibration model, and are difficult to determine accurately. In general, a certain number of pixels overlap between adjacent STIs for mosaicking based on the satellite design. We chose the middle TDI-CCD as the primary detector and calculated the internal parameters from the middle to two adjacent TDI-CCD detectors individually. Therefore, the internal parameters of the previous TDI-CCD detector can influence the priori geometric constraint for the next TDI-CCD detector, based on the overlapping pixels between adjacent STIs. Tie points between the overlapping regions of adjacent STIs can then be matched, as shown in Fig. 5(b), and their WGS84 object space coordinates can be obtained using the method shown in Fig. 3 based on the detector-calibrated internal parameters of the previous TDI-CCD. Because of the tiny intersection angle between overlapping pixels, the second version of the ASTER GDEM (GDEM2) is able to meet the requirement of the forward intersection. Because the relative geometric relationship of the adjacent TDI-CCD detectors in the overlapping regions is relatively stable along the track, the tie points should cover the entire overlapping region to guarantee the maximum number of matched tie points and stability of the estimation.
Because regions usually overlap in narrow stripes, distortions therein can be described by the simplified internal model using only given in Eq. (10), and can calculated based on geometric constraints of the tie points. Although the internal parameters can only compensate for distortion in the overlapping narrow stripes, the calculated values of and can be directly used to define the internal parameters of the next single TDI-CCD detector to be calibrated. Then, we need only calculate the other internal parameters and based on the self-calibration model.
and can be calculated from three stereoscopic images, and theoretically the results should be the same; in practice, we use the average value. For application of the calculated and , the image coordinate system of the secondary STIs on either side of the primary STI have different horizontal directions, as shown in Fig. 5(b).
Based on the least squares estimation method, we can obtain the normal equation of Eq. (9):
Because the number of unknown ground plane coordinates, , is much greater than the internal parameter, , we simplify Eq. (11) by first calculating the current correction, , using the elimination method, as
We can therefore define the terms and . Then, can be obtained by forward intersection based on the updated . Finally, we proceed iteratively until is a very small positive number. To make the iterative process converge faster, GDEM2 can be used to provide reference elevation information for the first several iterations. The stepwise parameter estimation method takes full advantage of the a priori geometric constraint and thus increases the stability of self-calibration. Furthermore, the mosaicking accuracy of all STIs can be guaranteed based on the calibrated internal parameters and , and this accuracy is very important during post-processing for generation of the whole (stitched) image.
The proposed parameters estimation method has no strict requirement for the initial values to achieve the optimal values. In engineering applications, the initial values for the on-orbit calibration can use the on-ground calibration results before launch or the initial design parameters [4, 7].
4. Experiments and discussion
4.1 Study area and data source
Launched in 2014, the Gaofen-2 (GF2) remote sensing satellite is a Chinese high-resolution optical satellite equipped with a double-camera pushbroom imaging system with a resolution of 0.81 m and a 45-km strip width. The focal plane of the single camera is composed of five collinear TDI-CCD detectors within the FOV. The width of each STI is 6,144 pixels and there are 380 overlapping pixels between adjacent STIs for mosaicking purposes, as shown in Fig. 6. Real and simulated data were used in experiments to verify the reliability of the proposed approach. Actual stereoscopic images of panchromatic sensor A, captured on three different tracks by side-imaging of the same region, and which are panchromatic with five collinear STIs after radiometric correction , were are used to verify the effectiveness of the self-calibration method. Because GF2 lacks same-track stereoscopic imaging capability, and stereoscopic images taken on the same track by other satellites with appropriate overlapping regions are also not presently available, we simulated stereoscopic images based on real GF2 satellite auxiliary data, the calibrated camera parameters provided for panchromatic sensor A, and the reference DOM and DEM. Table 1 provides detailed information about the stereoscopic images.
The DOM and DEM obtained from the WorldView3 satellite were used as reference data for self-calibration, geometric accuracy evaluation and simulation. The resolution of the DOM is 0.3 m and its planar accuracy is 0.3 m based on the root mean square (RMS) method; the DEM has a resolution of 2 m and height accuracy of 0.5 m by RMS. The GCPs for geometric accuracy assessment can be obtained from the reference data by automatic matching using the scale-invariant feature transform (SIFT) algorithm . The matching accuracy was better than 0.3 pixels. The initial calibration parameters were based on the initial design parameters of the camera.
4.2 Results and discussion
4.2.1 Self-calibration using stereoscopic images on different tracks
The STIs corresponding to the three stereoscopic images exhibit an appropriate triple overlapping relationship, as shown in Fig. 7. We selected the NO. 3 TDI-CCD as the primary detector, and automatically matched GCPs from the DOM as an absolute control to estimate the installation angles of image 2 and the internal parameters of the primary TDI-CCD detector. Assuming that a high-accuracy DEM is available for estimation of the relative installation angles of images 1 and 3, tie points around the GCPs were automatically matched. In Fig. 7, the discrete blue points represent the GCPs and the dense red points represent the matched tie points. After determining the relative coordinate system of the three stereoscopic images, the self-calibration model can be used for calibration of the internal parameters of the secondary TDI-CCD detectors. Then, tie points in the triple-overlap regions of the corresponding STIs and tie points between adjacent STIs were matched. Because the five TDI-CCD detectors of the camera on the GF2 satellite are collinear, the matched tie points on each STI are also collinear in the whole image to guarantee consistent imaging time.
The results of the calculated internal parameters of the primary sensor, TDI-CCD 3, and the installation angles of the stereoscopic images are shown in Table 2. The calibrated installation angles of the three stereoscopic images have obvious differences, probably due to changes in the installation angle during the long imaging time interval. However, with the aid of the accurately estimated internal parameters of primary TDI-CCD 3, the relative coordinate system of the stereoscopic images can be accurately determined from the calibrated installation angles.
After accurately determining the relative coordinate systems, the internal parameters of the secondary TDI-CCD detector can be calibrated via the self-calibration model. We can thus accurately calibrate the whole camera based only on localized GCPs covering a single STI, which greatly reduces the demand of creating a calibration field.
Figure 8 shows the residual errors caused by internal LOS distortion of each TDI-CCD detector after external calibration. The relatively smooth internal LOS distortion on the edge of each STI guarantees the accuracy of the first internal parameters and that are estimated for the secondary TDI-CCD detectors based on the simplified internal model in Eq. (10). Then, the other internal parameters and can be estimated.
Table 3 shows the positioning accuracy before and after internal calibration for each STI, which was evaluated based on the reference DOM and DEM. The “after-S” represents the calibration accuracy by the self-calibration method, while “after-T” represents the calibration accuracy by the traditional calibration method based on GCPs. Residual errors caused by internal distortions are still significant after external calibration, but after internal self-calibration the positioning accuracy of each STI improves to better than 1.5 pixels due to the excellent performance of the proposed self-calibration model. In addition, we can see that the calibration accuracies of the secondary TDI-CCD detector based on the self-calibration method and the traditional calibration method are very similar, indicating that the proposed calibration approach has similar calibration ability to traditional methods.
Stepwise internal parameter estimation guarantees mosaic accuracy; to evaluate its effectiveness, mosaic accuracy was measured based on the consistency of the positioning accuracy of matched tie points in the overlapping regions between adjacent STIs. Using the rigorous imaging model and calibrated parameters for each TDI-CCD detector, the object space coordinates of the tie points can be obtained by forward intersection, and the deviation between two object space coordinates of the tie points thus obtained will reflect the mosaic accuracy. The mosaic accuracy is shown in Table 4. The planar mosaic accuracy is improved to better than 0.3 pixels using the proposed method, indicating that seamless mosaicking can be achieved using the relative geometric information determined from the internal parameters. The accuracy of the whole (seamlessly stitched) image is described in Table 3. There was no loss of accuracy after the stitching process, indicating geometric consistency of the calibrated internal parameters.
Using a few GCPs to eliminate the exterior systematic offset of the positioning based on the affine rational polynomial coefficient (RPC) model , the internal accuracy of other images can be evaluated to verify the effectiveness of the calibrated internal parameters, the results of which are shown in Table 5.
Three images were selected, covering three different high-accuracy calibration fields located in the Chinese cities of Dongying, Zibo and Jimo. Four GCPs were matched from the reference DOM and DEM, and used with the affine RPC model to eliminate exterior systematic offset; the other 60 GCPs were then used to evaluate the relative internal accuracy. As shown in Table 5, the positioning accuracy obtained using a few GCPs was better than 1.5 pixels after calibration, and imaging time and area had no influence on the relative internal accuracy.
4.2.2 Self-calibration using stereoscopic images on the same track
Because GF2 lacks the capability of stereoscopic imaging on the same track, three stereoscopic GF2 images of the same area on the same track were simulated, with each stereoscopic image composed of five collinear STIs. The overlap among the stereoscopic images is shown in Fig. 9, and it is apparent that each STI has the appropriate triple overlap regions for self-calibration. The dislocation across the track was caused by different installation roll angles and the tilted orbit, and the same imaging area was obtained through different installation pitch angles for forward, downward and backward imaging. The simulated installation angles are shown in Table 6. We aimed to confirm that this type of overlap on the same track meets the requirements of the proposed self-calibration method.
The same processing flow was utilized to ensure the high-accuracy calibration. We selected the No. 3 TDI-CCD as the primary detector, calibrated its internal parameters and the installation angles for image 1 based on discrete GCPs, and then determined the installation angles of image 2 and 3 based on the calibrated internal parameters. The internal parameters of the primary TDI-CCD are shown in Table 6, and the installation angles of the stereoscopic images are shown in Table 7. In Table 6, although the calibrated internal parameters of the primary TDI-CCD 3 detector have obvious deviations from the simulated values (comparing calibrated values with simulated values), the positioning accuracy after calibration can be much higher due to concordant installation angles and internal parameters based on the rigorous geometric imaging model, as shown in Table 8. This high accuracy is due to the strong correlation between the installation and internal directional angles. As shown in Table 7, although the calibrated installation angles have obvious deviations from the simulated values (comparing calibrated and simulated values), the relative installation accuracy of the three stereoscopic images can be guaranteed based on the high consistency of the difference between their calibrated and simulated installation angels (shown as Image 2 - Image 1, Image 3 - Image 1). That consistency was due to the installation angles of the three stereoscopic images, which were based on the same highly accurate calibration of the primary TDI-CCD internal parameters. Accurately calibrated primary TDI-CCD internal parameters guarantee the accuracy of the three external relative reference coordinate systems, reflecting their spatial relationship and meeting the requirement for self-calibration.
After determining the relative reference coordinate systems of the stereoscopic images, the secondary TDI-CCD detectors can be calibrated using the proposed self-calibration approach. Tie points between adjacent STIs were used for the estimation of and for each secondary TDI-CCD, in the order 2, 4, 1 and 5 from the middle to each side. Meanwhile, the tie points in the triple-overlap region of each STI can be used for estimation of and for the secondary TDI-CCD detectors. Table 8 shows the positioning accuracy of each STI before and after calibration of the internal distortion. After calibration of the primary TDI-CCD detector based on discrete GCPs, and internal self-calibration of the secondary TDI-CCD detectors based on tie points, the positioning accuracy of the five TDI-CCD detectors was better than 0.3 pixels and was without systematic errors, which were completely eliminated by the proposed self-calibration model. Besides, the calibration accuracies of the secondary TDI-CCD detector based on the self-calibration method and the traditional calibration method are very similar, indicating that the proposed calibration approach has similar calibration ability to traditional methods.
The effectiveness of the proposed internal parameter estimation method for improving and maintaining the mosaicking accuracy of two adjacent STIs can be verified by comparing the positioning accuracy of the homonymy points on the regions of overlap between two adjacent STIs. We transformed the positioning residual error into pixel residual error, and the results are shown in Table 9. The relative geometric relationship between adjacent TDI-CCD detectors was determined accurately after calibration. Therefore, based on the relative geometric information, we can achieve seamless mosaicking of whole (stitched) images without local correction or accuracy loss. Comparing the calibration accuracies of the single TDI-CCD detector and the whole image, we can also see that there is no loss of accuracy after the stitching process, which indicates the geometric consistency of the calibrated internal parameters.
Sharing the same internal distortion is an important precondition for the proposed self-calibration approach, as it is necessary for obtaining an accurate relative coordinate system. Images captured within a shorter interval would probably have more consistent internal distortion. Therefore, using stereoscopic images on the same track confers a significant advantage; however, this method demands high maneuverability of the satellite. The internal distortion is relatively stable on orbit, and when taking stereoscopic images on different tracks the imaging interval can be shortened by adjusting the roll angle, thus attaining high-accuracy self-calibration.
Based on theoretical analysis and experimental results, the greatest advantage of our proposed self-calibration approach is the use of a small number of absolute GCPs covering the primary STI, which allows highly accurate calibration of the whole image; this greatly reduces the demand for a large-scale calibration field and is thus more efficient in terms of time, effort and cost. Based on the self-calibration model and the stepwise parameter estimation method, the a priori geometric constraints of multi-view stereoscopic images are used efficiently for calibration. Both the internal relative accuracy of the STI and the mosaicking accuracy between adjacent STIs can be guaranteed. Therefore, there is no accuracy loss after stitching the STIs together to generate the whole (stitched) image. To avoid error accumulation caused by successive computation from the primary STI to two sides without an absolute control, the internal LOS accuracy for both sides of the TDI-CCD detectors should be guaranteed by using a good distribution of GCPs for the primary TDI-CCD and tie points for the secondary TDI-CCD. If a small number of GCPs can be obtained in each overlapping region between adjacent STIs, the internal parameters and are more precisely estimated and the calibration results are more reliable.
In this paper, a simple and feasible self-calibration scheme was proposed that uses a small number of absolute GCPs covering the primary STI to achieve high-accuracy calibration of the whole image. This method can greatly reduce dependence on a highly accurate calibration field. A geometric self-calibration model based on three-view stereoscopic images and a stable high-accuracy internal parameter stepwise estimation method were proposed, and the conditions necessary for application of the self-calibration model were fully analyzed and discussed. This self-calibration method is appropriate for most high-resolution multi-TDI-CCD optical satellites, and its effectiveness has been verified for stereoscopic imaging both on different tracks and on the same track.
National Natural Science Foundation of China (Project Nos. 91438203, 91438111); National Basic Research Program of China 973 Program (Project Nos. 2014CB744201); Program for New Century Excellent Talents in University.
References and links
2. J. Grodecki, “IKONOS geometric calibrations,” Proceedings of the ASPRS 2005 Annual Conference (2005).
4. M. Wang, Y. F. Cheng, X. L. Chang, S. Y. Jin, and Y. Zhu, “On-orbit geometric calibration and geometric quality assessment for the high-resolution geostationary optical satellite gaofen4,” ISPRS J. Photogramm. Remote Sens. 125, 63–77 (2017). [CrossRef]
5. A. Gruen, S. Kocaman, and K. Wolff, “Calibration and validation of early ALOS/PRISM images,” J. Jpn. Soc. Photogram. Rem. Sens. 46(1), 24–38 (2007).
6. T. Tadono, M. Shimada, H. Murakami, and J. Takaku, “Calibration of PRISM and AVNIR-2 onboard ALOS—Daichi,” IEEE Trans. Geosci. Remote Sens. 47(12), 4042–4050 (2009). [CrossRef]
7. M. Wang, B. Yang, F. Hu, and X. Zang, “On-Orbit geometric calibration model and its applications for high-resolution optical satellite imagery,” Remote Sens. 6(12), 4391–4408 (2014). [CrossRef]
8. X. Shao, M. M. Eisa, Z. Chen, S. Dong, and X. He, “Self-calibration single-lens 3d video extensometer for high-accuracy and real-time strain measurement,” Opt. Express 24(26), 30124–30138 (2016). [CrossRef] [PubMed]
13. O. D. Faugeras, Q. T. Luong, and S. J. Maybank, “Camera self-calibration: Theory and experiments,” Computer Vision—ECCV ,588(12), 321–334 (1992).
14. E. Malis and R. Cipolla, “Camera self-calibration from unknown planar structures enforcing the multiview constraints between collineations,” IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1268–1272 (2002). [CrossRef]
15. K. C. Di, Y. L. Liu, B. Liu, M. Peng, and W. M. Hu, “A Self-Calibration Bundle Adjustment Method for Photogrammetric Processing of Chang E-2 Stereo Lunar Imagery,” IEEE Trans. Geosci. Remote Sens. 52(9), 5432–5442 (2014). [CrossRef]
16. M. T. Zheng, Y. J. Zhang, J. F. Zhu, and X. D. Xiong, “Self-Calibration adjustment of CBERS-02B long-strip imagery,” IEEE Trans. Geosci. Remote Sens. 53(7), 3847–3854 (2015). [CrossRef]
17. J. M. Delevit, D. Greslou, V. Amberg, C. Dechoz, F. de Lussy, L. Lebegue, C. Latry, S. Artigues, and L. Bernard, “Attitude assessment using Pleiades-HR capabilities,” in Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (2012), 525–530. [CrossRef]
18. G. Zhang, K. Xu, Q. J. Zhang, and D. R. Li, “Correction of Pushbroom Satellite Imagery Interior Distortions Independent of Ground Control Points,” Remote Sens. 10(2), 98 (2018). [CrossRef]
19. M. Wang, Y. Zhu, S. Y. Jin, J. Pan, and Q. S. Zhu, “Correction of zy-3 image distortion caused by satellite jitter via virtual steady reimaging using attitude data,” ISPRS J. Photogramm. Remote Sens. 119, 108–123 (2016). [CrossRef]
20. J. Li, F. Xing, T. Sun, and Z. You, “Efficient assessment method of on-board modulation transfer function of optical remote sensing sensors,” Opt. Express 23(5), 6187–6208 (2015). [CrossRef] [PubMed]
21. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis. 60(2), 91–110 (2004). [CrossRef]
22. C. S. Fraser and H. B. Hanley, “Bias compensation in rational functions for IKONOS satellite imagery,” Photogramm. Eng. Remote Sensing 69(1), 53–57 (2003). [CrossRef]