All-parameter calibration method of the on-orbit multi-view dynamic photogrammetry system

Qilin Liu; Qilin Liu; Mingli Dong; Peng Sun; Bixi Yan; Jun Wang; Lianqing Zhu; Lianqing Zhu

doi:10.1364/OE.482386

1. Introduction

As space technology has been leaping forward, a wide variety of space structures (e.g., space antenna, space telescope, and space solar array [1–3]) are rapidly advancing towards large-scale, high-precision and deployable space structures. In order to verify the reliability of the structure and monitor its functional performance, all large space structures (LSS) will undergo a series of rigorous ground-based or space-based tests. PG shows the advantages of non-contact, large-scale, high-precision, multi-point simultaneous measuring, and others. It especially applies to the fast measurement of large structures movement and deformation [4]. As the space camera thermal control system is rapidly advancing [5], the PG system has been well applied to a series of LSS ground-based or space-based measurement tasks.

In ground-based tests, from 2009 to 2017, the European Space Research and Technology Centre developed a semi-automatic PG system working at a low temperature (−170 °C) to examine the structure and mirrors of the Herschel Space Telescope [6–8]. The measurement accuracy was determined as 50 µm, 1 σ. From 2007 to 2020, the National Aeronautics and Space Administration (NASA) performed several vacuum and low-temperature tests on the structure [9–14] and the optical system [15,16] of the James Webb Space Telescope using the PG system. The last measurement uncertainty was obtained as ±0.100 mm, 2 σ, per axis, per target point [17]. In 2018, the Beijing Institute of Spacecraft Environment Engineering proposed an antenna distortion measurement system using the PG system [18]. The thermal deformation of the ϕ4.2 m satellite deployable mesh antenna was examined with an accuracy of 50 µm/ 5 m in a vacuum at low temperatures.

In the above ground-based tests, the large cryogenic executable structure has been adopted to assist the networking of multiple cameras, and high-precision thermal control boxes have been employed to ensure the functional performance of cameras at low temperatures. In this way, the PG system can maintain high measurement accuracy. It is noteworthy that such a complex ground-based PG system is not feasible in on-orbit measurement tasks. Moreover, the ground-based test can still not provide a prediction model for accurately predicting LSS on-orbit functional performance.

In space-based tests, as early as 1993, during the first servicing mission of the Hubble Space Telescope, NASA performed the three-dimensional (3D) feature point measurements of the solar panels of the Hubble Space Telescope using four cameras carried by the space shuttle [19]. In 1996, during docking missions of the Space Shuttle to the Russian Space Station Mir, NASA performed modal tests on the solar panels of the Mir space station using the PG system carried by the space shuttle [20,21]. Specifically, six pre-calibrated cameras were adopted to examine the first five natural frequencies, damping ratios, and mode shapes of the Mir Kvant-II solar array. The latest application of on-orbit PG was in January 2022. NASA Langley Research Center collaborated with the Johnson Space Center to develop a PG system using International Space Station (ISS) cameras. The structural parameters (e.g., the array modal shape, 3-axis acceleration, and velocity of the Roll-Out Solar Array) were examined using five ISS cameras [22].

No specific measurement accuracy has been given in the above application of the on-orbit PG. The CIP is pre-calibrated on the ground. The CEP orientation employs the known spatial reference information (KSRI) in space (e.g., the design coordinates of structural nodes of the spacecraft). Since the KSRI and the pre-calibrated PG system will be subjected to notoriously severe force and thermal environment variation during spacecraft launching, their on-orbit deploying and working [23,24], the pre-calibrated information may change. Thus, the spatial reconstruction results and the accuracy evaluation criteria of PG are unreliable. As a result, more stable reference information in space should be adopted to help the PG system complete the on-orbit calibration.

The star sensor calibration and attitude measurement accuracy can achieve arcsecond accuracy using stars [25]. However, the star is almost infinitely far away from the image plane. It cannot provide the spatial position reference for CEP. The scale bar, which is usually a rod made of carbon fiber or invar alloy with a very low coefficient of thermal expansion, can provide a relative position reference for CEP. There are two photogrammetric retro-reflective targets fixed at each end of the bar. The distance between the two targets can be examined using a high-precision instrument (e.g., Laser Tracker) [26]. For its thermal and structural stability, the scale bar is relatively easy to deploy in orbital missions. Moreover, stars and scale bars can provide constraints for the CIP and LDP calibration.

Accordingly, this study proposes the application of stars and scale bars to calibrate all parameters of the OMDPS. Firstly, to solve the no-KSRI calibration problem, a full-parameter calibration model, which can decouple the correlation between interior and exterior parameters of multiple cameras, is designed using stars and scale bars. Secondly, a multi-camera bundle adjustment algorithm is built to support the fusion of multi-type observed data (e.g., star, scale bar target, and scale bar length).

2. Mathematical model

2.1 Camera imaging model

In a camera system, the principal point does not entirely coincide with the image plane center, and the camera lens cannot achieve ideal perspective imaging. The camera imaging model used in this paper is shown in Fig. 1.

Fig. 1. Camera imaging model.

Download Full Size | PDF

As depicted in Fig. 1, O is the center of the image plane. The location of the principal point on the image plane is O′ (x_p, y_p). The principal distance is f. D (X, Y, Z) is the spatial coordinate of a scale bar target. d (x_d, y_d) is the scale bar target projected coordinate in the image plane. d_l is the scale bar target ideal pinhole projective coordinate in the image plane. S (α, δ) is the right ascension and declination coordinate of a star in the celestial coordinate system. s (x_s, y_s) is the star projected coordinate in the image plane. s_l is the star ideal pinhole projective coordinate in the image plane. [Δx, Δy]^T are the distortions that denote the deviation of the image point from the ideal pinhole projective image point and are expressed by the following polynomials:

(1)$$\begin{aligned}&\Delta x = \bar{x}\left( {K_1r^2 + K_2r^4 + K_3r^6} \right) + P_1\left( {2{\bar{x}}^2 + r^2} \right) + 2P_2\overline {xy} + b_1\bar{x} + b_2\bar{y} \\& \Delta y = \bar{y}\left( {K_1r^2 + K_2r^4 + K_3r^6} \right) + P_2\left( {2{\bar{y}}^2 + r^2} \right) + 2P_1\overline {xy} \\& \bar{x} = x-x_p{\kern 1cm} \bar{y} = y-y_p{\kern 1cm} r = \sqrt {{\bar{x}}^2 + {\bar{y}}^2}\end{aligned}$$

where, (x, y) are the projected coordinates of scale bar targets and stars in the image; K₁, K₂, and K₃ express the radial distortion parameters; P₁ and P₂ represent the tangential distortion parameters; b₁ and b₂ are the affine distortion parameters in the image plane; r is the radial distance, representing the distance between the image point and the principal point; The image coordinates (e.g., stars and scale bar targets) used subsequently need to be corrected by Eq. (1) to conform to the ideal pinhole perspective imaging model; The CIP (f, x_p, y_p) and the LDP (K₁, K₂, K₃, P₁, P₂, b₁, b₂) are related to the above model.

2.2 Star imaging model

Stars in the celestial coordinate system can be expressed in the right ascension and declination (α, δ). Based on the star sensor imaging model [25], when a camera with attitude matrix R is in the celestial coordinate system, a star S (α, δ) imaging model is shown in Fig. 1 and expressed as follows:

(2)$$\begin{aligned}x_s &= f\displaystyle{{R_{11}\cos \alpha \cos \delta + R_{12}\sin \alpha \cos \delta + R_{13}\sin \delta } \over {R_{31}\cos \alpha \cos \delta + R_{32}\sin \alpha \cos \delta + R_{33}\sin \delta }} + x_p-\Delta x \\y_s &= f\displaystyle{{R_{21}\cos \alpha \cos \delta + R_{22}\sin \alpha \cos \delta + R_{23}\sin \delta } \over {R_{31}\cos \alpha \cos \delta + R_{32}\sin \alpha \cos \delta + R_{33}\sin \delta }} + y_p-\Delta y\end{aligned} $$

The imaging equation of any star (c-th camera, i-th star) can be expanded by the first-order Taylor formula, and the linearized correction equation for an observed star image point is:

(3)$$\mathop {v{s_{c,i}}}\limits_{(2 \times 1)} + \mathop {l{s_{c,i}}}\limits_{(2 \times 1)} = \mathop {{J_{c,i}}}\limits_{(2 \times 13)} \mathop {{{\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}} \over \delta } }_c}}\limits_{(13 \times 1)}$$

where, vs denotes the residual vector of a star image point that is defined by the disparity vector between the “true” (without error) star image point coordinate and the measured star image point coordinate; ls represents the reduced observation vector that is defined by the disparity vector between the measured star image point coordinate and the computed star image point coordinate using the approximate camera parameters. In Eq. (2), the x_s and y_s functions are defined by the CIP, LDP and camera attitude parameters (φ, ϖ, κ) in CEP (φ, ϖ, κ, T_x, T_y, T_z). The partial derivatives of these two functions with respect to parameters above can be calculated. Jacobian matrix J can be constructed by arranging the partial derivatives. $\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\frown$}} \over \delta }$ are the corrections of the camera parameters mentioned above.

2.3 Scale bar target imaging model

In the on-orbit environment, there is no known spatial position information of any target in the celestial coordinate system to assist the spatial position calculation of all cameras. Therefore, OMDPS adopts a multi-camera relative position model to realize the spatial positioning of on-orbit cameras in the celestial coordinate system. Through the relative distance between two targets on the scale bar and the matched target image points in the images of four cameras, the spatial positions of the other three cameras relative to the reference camera are constrained. The position parameter of the reference camera can be set to (0, 0, 0) by translating its coordinate system center to the celestial coordinate system center.

A collinear equation is established with the 3D coordinates of scale bar targets in space and all parameters of all cameras except for the reference camera position parameters as the unknown variables [26]. The scale bar target D (X, Y, Z) imaging model is shown in Fig. 1. The imaging model of the target image point d_c,k $({x_{{d_{c,k}}}},{y_{{d_{c,k}}}})$ of the k-th scale bar target D_k (X_k, Y_k, Z_k) in the c-th camera is expressed as follows:

(4)$$\begin{aligned} {x_{{d_{c,k}}}} &={-} {f_c}\frac{{{R_{{c_{11}}}}({X_k} - {T_{xc}}) + {R_{{c_{12}}}}({Y_k} - {T_{yc}}) + {R_{{c_{13}}}}({Z_k} - {T_{zc}})}}{{{R_{{c_{31}}}}({X_k} - {T_{xc}}) + {R_{{c_{32}}}}({Y_k} - {T_{yc}}) + {R_{{c_{33}}}}({Z_k} - {T_{zc}})}} + {x_{pc}} - \Delta {x_{c,k}}\\ {y_{{d_{c,k}}}} &={-} {f_c}\frac{{{R_{{c_{21}}}}({X_k} - {T_{xc}}) + {R_{{c_{22}}}}({Y_k} - {T_{yc}}) + {R_{{c_{23}}}}({Z_k} - {T_{zc}})}}{{{R_{{c_{31}}}}({X_k} - {T_{xc}}) + {R_{{c_{32}}}}({Y_k} - {T_{yc}}) + {R_{{c_{33}}}}({Z_k} - {T_{zc}})}} + {y_{pc}} - \Delta {y_{c,k}} \end{aligned}$$

where, (X_k, Y_k, Z_k) is the space coordinates of the k-th scale bar target in the reference camera coordinate system;$({x_{{d_{c,k}}}},{y_{{d_{c,k}}}})$ represents the k-th scale bar target point actual projected coordinates in the c-th camera image; f_c denotes the principal distance of the c-th camera; R_c represents the attitude matrix of the c-th camera in the celestial coordinate system; (T_xc, T_yc, T_zc) represents the relative space position of the c-th camera in the reference camera coordinate system. When c = 1 (the reference camera), T_x₁ = T_y₁ = T_z₁ = 0; (x_pc, y_pc) represents the principal point of the c-th camera; [Δx_c,k, Δy_c,k]^T express the imaging distortion at the scale bar target image point of the k-th scale bar target captured by the c-th camera.

The imaging equation (Eq. (4)) of any target on the scale bars (c-th camera, n-th scale bar, and k-th scale bar target) can be expanded by the first-order Taylor formula, and the linearized correction equation for an observed scale bar target image point is expressed as follows:

(5)$$\mathop {v{p_{c,n,k}}}\limits_{(2 \times 1)} + \mathop {l{p_{c,n,k}}}\limits_{(2 \times 1)} = \mathop {{A_{c,n,k}}}\limits_{(2 \times 16)} \mathop {{\delta _c}}\limits_{(16 \times 1)} + \mathop {{B_{c,n,k}}}\limits_{(2 \times 3)} \mathop {{{\dot{\delta }}_{n,k}}}\limits_{(3 \times 1)}$$

where, vp denotes the residual vector of a target image point that is defined by the disparity vector between the “true” (without error) target image point coordinate and the measured target image point coordinate; lp represents the reduced observation vector that is defined by the disparity vector between the measured target image point coordinate and the computed target image point coordinate using the approximate camera parameters. In Eq. (4), the x_d and y_d functions are defined by the CIP, LDP, CEP and space coordinates of scale bar targets. The partial derivatives of these two functions with respect to parameters above can be calculated. Jacobian matrix A can be constructed by arranging the partial derivatives of these two functions with respect to camera parameters. Jacobian matrix B can be constructed by arranging the partial derivatives of these two functions with respect to target coordinates. δ are the corrections of the camera parameters mentioned above. $\dot{\delta }$ are the corrections of the target coordinates mentioned above.

2.4 Scale bar length model

To scale a photogrammetric measurement, at least one known distance must be present in the imagery. Only two photogrammetric retro-reflective targets are fixed at each end of the scale bar. The n-th scale bar length DIS_n equation determined by the right scale bar target D_n,r (X_n,r, Y_n,r, Z_n,r) and the left scale bar target D_n,l (X_n,l, Y_n,l, Z_n,l) is as follows:

(6)$$DI{S_n} = \sqrt {{{({X_{n,r}} - {X_{n,l}})}^2} + {{({Y_{n,r}} - {Y_{n,l}})}^2} + {{({Z_{n,r}} - {Z_{n,l}})}^2}}$$

The distance equation of a target pair on any scale bars (n-th scale bar, left target l, and right target r) can be expanded by the first-order Taylor formula, and the linearized correction equation for a spatial distance of a target pair is expressed as follows:

(7)$$\mathop {v{d_n}}\limits_{(1 \times 1)} + \mathop {l{d_n}}\limits_{(1 \times 1)} = \mathop {{C_{n,l}}}\limits_{(1 \times 3)} \mathop {{{\dot{\delta }}_{n,l}}}\limits_{(3 \times 1)} + \mathop {{C_{n,r}}}\limits_{(1 \times 3)} \mathop {{{\dot{\delta }}_{n,r}}}\limits_{(3 \times 1)}$$

where, vd denotes the residual vector of a scale bar length that is defined by the disparity vector between the “true” (without error) scale bar length and the measured scale bar length; ld represents the reduced observation vector that is defined by the disparity vector between the measured scale bar length and the computed scale bar length using the approximate scale bar target coordinates. In Eq. (6), the DIS function is defined by the space coordinates of scale bar targets. The partial derivatives of the DIS function with respect to each target coordinate can be calculated. Jacobian matrix C can be constructed by arranging the partial derivatives. $\dot{\delta }$ are the corrections of the target coordinates mentioned above.

3. Full-parameter bundle adjustment algorithm

From the constraint equations determined by the above model, the CIP, LDP, and camera attitude parameters in the CEP of all cameras are jointly constrained by stars, scale bar targets and scale bar lengths. The camera relative position parameters in CEP are constrained by scale bar targets and scale bars lengths. With Eqs. (3), (5) and (7), the linearized correction equations of three different data types can be combined to build a large-scale extended correction equation which contains star image points, scale bar target image points and scale bar lengths constraints. The matrix elements of the extended correction equation are expressed in Eq. (8).

(8)$$\left[ {\begin{array}{@{}c@{}} \begin{array}{@{}c@{}} v{s_{1,1}}\\ \vdots \\ v{s_{1,i}}\\ \vdots \\ v{s_{c,1}}\\ \vdots \\ v{s_{c,i}} \end{array}\\ {v{p_{1,1,r}}}\\ \begin{array}{@{}c@{}} v{p_{1,1,l}}\\ \vdots \\ v{p_{1,n,r}}\\ v{p_{1,n,l}}\\ \vdots \\ v{p_{c,1,r}} \end{array}\\ {v{p_{c,1,l}}}\\ \vdots \\ {v{p_{c,n,r}}}\\ {v{p_{c,n,l}}}\\ {v{d_1}}\\ \vdots \\ {v{d_n}} \end{array}} \right] + \left[ {\begin{array}{@{}c@{}} \begin{array}{@{}c@{}} l{s_{1,1}}\\ \vdots \\ l{s_{1,i}}\\ \vdots \\ l{s_{c,1}}\\ \vdots \\ l{s_{c,i}} \end{array}\\ {l{p_{1,1,r}}}\\ \begin{array}{@{}c@{}} l{p_{1,1,l}}\\ \vdots \\ l{p_{1,n,r}}\\ l{p_{1,n,l}}\\ \vdots \\ l{p_{c,1,r}} \end{array}\\ {l{p_{c,1,l}}}\\ \vdots \\ {l{p_{c,n,r}}}\\ {l{p_{c,n,l}}}\\ {l{d_1}}\\ \vdots \\ {l{d_n}} \end{array}} \right] = \left[ {\begin{array}{@{}ccccccccc@{}} {{J_{1,1}}}&0&0&0&0&0&0&0\\ \vdots & \vdots &0& \vdots &0&0&0&0\\ {{J_{1,i}}}&0& \vdots &0& \vdots &0&0&0\\ 0& \ddots &0& \vdots &0& \vdots &0&0\\ \vdots &0&{{J_{c,1}}}&0& \vdots &0& \vdots &0\\ 0& \vdots & \vdots & \vdots &0& \vdots &0& \vdots \\ 0&0&{{J_{c,i}}}&0& \vdots &0& \vdots &0\\ {{A_{1,1,r}}}&0&0&{{B_{1,1,r}}}&0& \vdots &0& \vdots \\ {{A_{1,1,l}}}&0&0&0&{{B_{1,1,l}}}&0& \vdots &0\\ \vdots & \vdots &0& \vdots &0& \ddots &0& \vdots \\ {{A_{1,n,r}}}&0& \vdots &0& \vdots &0&{{B_{1,n,r}}}&0\\ {{A_{1,n,l}}}&0&0& \vdots &0& \vdots &0&{{B_{1,n,l}}}\\ 0& \ddots &0&0& \vdots &0& \vdots &0\\ 0&0&{{A_{c,1,r}}}&{{B_{c,1,r}}}&0& \vdots &0& \vdots \\ \vdots &0&{{A_{c,1,l}}}&0&{{B_{c,1,l}}}&0& \vdots &0\\ 0& \vdots & \vdots & \vdots &0& \ddots &0& \vdots \\ 0&0&{{A_{c,n,r}}}&0& \vdots &0&{{B_{c,n,r}}}&0\\ \vdots &0&{{A_{c,n,l}}}&0&0& \vdots &0&{{B_{c,n,l}}}\\ 0& \vdots &0&{{C_{1,r}}}&{{C_{1,l}}}&0& \vdots &0\\ 0&0& \vdots &0&0& \ddots &0&0\\ 0&0&0&0&0&0&{{C_{n,r}}}&{{C_{n,l}}} \end{array}} \right]\left[ \begin{array}{c} {\delta_1}\\ \vdots \\ {\delta_c}\\ {{\dot{\delta }}_{1,r}}\\ {{\dot{\delta }}_{1,l}}\\ \vdots \\ {{\dot{\delta }}_{n,r}}\\ {{\dot{\delta }}_{n,l}} \end{array} \right]$$

where, δ are the corrections of all camera parameters; $\dot{\delta }$ are the corrections of all scale bar target coordinates.

In Eq. (8), the space position of the reference camera is located at the center of the celestial coordinate system by default, and does not participate in bundle adjustment. Excluding three position parameters of the reference camera position, 61 camera parameters and 3kn scale bar target coordinate parameters should be optimized. The implicit expression of Eq. (8) is expressed as follows:

(9)$$\mathop {{v_{all}}}\limits_{(2ic + 2nkc + n) \times 1} + \mathop {{l_{all}}}\limits_{(2ic + 2nkc + n) \times 1} = \mathop {{H_{all}}}\limits_{((2ic + 2nkc + n) \times (61 + 3kn))} \mathop {{\delta _{all}}}\limits_{((61 + 3kn) \times 1)}$$

where, the subscripts (i, c, k, n) denote the i-th star in the c-th (c = 1, 2, 3, 4) camera and the k-th (k = r, l) target of the n-th scale bar in the c-th camera; v_all denotes the residual vector of star image points, scale bar target image points, and scale bar lengths. It is defined by the disparity vector between the “true” (without error) value and the measured value of the three types of data mentioned above. l_all represents the reduced observation vector of three types of data. It is defined by the disparity vector between the measured value and the computed value using the approximate camera parameters and scale bar target coordinates. Jacobian matrix H_all is constructed by Jacobian matrix J, A, B, and C. H_all contains the partial derivatives of Eqs. (2), (4), and (6) with respect to all parameters (e.g., all camera parameters and scale bar target coordinates). δ_all are the corrections of all camera parameters and scale bar target coordinates.

The corrections equation for all camera parameters and scale bar target coordinates calculated by none-linear least squares is written as follows:

(10)$${\delta _{all}} = {({H_{all}}^T{H_{all}})^{ - 1}}{H_{all}}^T{l_{all}}$$

Since the bundle adjustment model involves multiple observed data and parameters in different magnitudes, the magnitude difference between the elements in H_all is also excessively large, thus causing the matrix ill-conditioned problem. To unify the magnitude of matrix elements in the iterative process, the Jacobian matrix is normalized in different two-norms for each column, as is shown in Eq. (11).

(11)$$\begin{array}{c} {H_{norm}} = {H_{all}}\underset{(61+3 k n) \times(61+3 n k)}{N^{-1}}\\ {\delta _{norm}} = {({H_{norm}^T{H_{norm}}} )^{ - 1}}\;\;H_{norm}^T{l_{all}} \end{array}$$

N is a two-norm diagonal matrix computed from each column of H_all.

(12)$${\delta _{all}} = \underset{(61+3 k n) \times(61+3 n k)}{N^{-1}}{\delta _{norm}}$$

Through Eq. (12), we can get the right parameter correction matrix δ_all.

Different types of observed data (e.g., stars, scale bar targets and scale bar lengths) have different measurement accuracy, so the weights of the above data should be adjusted in the calculation. The prior standard deviations of the observation error of star image points, scale bar target image points, and scale bar lengths are set as σ_s, σ_p, and σ_l, respectively. The prior standard deviation of the scale bar length is taken as the unit weight, and the weight matrix participating in the bundle adjustment optimization can be implicitly expressed as:

(13)$$P = {\left[ {\begin{array}{ccc} {\mathop {{P_s}}\limits_{(2ic \times 2ic)} }&0&0\\ 0&{\mathop {{P_p}}\limits_{(3kn \times 3kn)} }&0\\ 0&0&{\mathop {{P_l}}\limits_{(n \times n)} } \end{array}} \right]_{(61 + 3kn) \times (2ic + 2nkc + n)}}$$

In Eq. (13), P_s, P_p, and P_l denote the diagonal matrix with diagonal elements σ_l²/σ_s², σ_l²/σ_p², and σ_l²/σ_l², respectively. Finally, the corrections equation is updated as follows:

(14)$${\delta _{all}} = {N^{ - 1}}{({H^T_{norm}}P{H_{norm}})^{ - 1}}{H^T_{norm}}{Pl_{all}}$$

By using a weight matrix to adjust the observation accuracy of multiple data, the problem of inaccurate calibration results caused by different data observation accuracy is solved. Lastly, the Levenberg-Marquardt (LM) algorithm is adopted to iteratively correct the estimated values of all parameters till the optimal result is obtained.

Theoretically, according to Eq. (2), one star can provide two observation equations for each camera. Each camera can successfully calibrate its CIP, LDP, and camera attitude parameters in CEP only by observing seven stars. According to Eqs. (4) and (6), each additional scale bar can provide the whole system with 16 observation equations of target image point coordinates, one distance constraint equation and six unknown parameters of target spatial coordinates. The relative position parameters in the CEP of four cameras can be calibrated only by observing one scale bar in the convergent view.

However, to further improve the calibration accuracy of the system parameters, more redundant observed data should be provided. Stars in each camera field of view should be extracted and identified as much as possible. Scale bars should be evenly distributed in the system convergent view with certain depth and attitude variations.

4. Experiment and discussion

4.1 Simulation experiments and results

A simulation experiment system is developed to verify the effectiveness of the proposed model and evaluate the performance of the system. The system comprises an observed data generation module, a camera imaging module, an algorithm module, and a data analysis module.

Specifically, the observed data generation module covers all-sky stars, scale bar targets, scale bar lengths, and spatial target array to be measured. The camera imaging module involves camera parameter configuration and camera imaging model with different observed data. The algorithm module includes star map recognition, image point matching of scale bar target, all parameter estimation method, system full-parameter bundle adjustment, image point matching of spatial target array to be measured, point cloud 3D reconstruction, and point cloud 3D registration. Furthermore, the data analysis module comprises functions such as full-parameter calibration accuracy analysis, error analysis of reconstructed point clouds, display of error distribution, etc. The composition diagram of OMDPS modules is illustrated in Fig. 2:

Fig. 2. The composition diagram of OMDPS simulation system modules.

Download Full Size | PDF

4.1.1 Simulation system and scene configuration

In accordance with the data of the Tycho-2 Satr Catalogue, the all-sky star vector coordinates are generated, and the imaging simulation is carried out for the star points below 5.2 VT magnitude within the field of view. Four scale bars are set, and the distance between the scale bar target pair is 1000 mm. The spatial target array to be measured is generated, which covers 2500 spatial targets.

In the camera imaging module, the HIKROBOT camera (MV-CH120-10UM/UC, HIKROBOT, Hang Zhou, China) serves as the reference device to configure the simulation parameters. The camera field of view is 35°, and the image resolution is 4096 × 3000. The image sensor model is Sony IMX304, 1.1” CMOS. The pixel size is 3.45 µm. The CEP, CIP, and LDP are set with the values of the real HIKROBOT cameras, as shown in Table 1.

Table 1. Camera parameters configuration in simulation scenario

View Table | View all tables in this article

Figure 3(a) presents the on-orbit working scenario of the OMDPS. The system can be deployed on the space station to form a multi-camera intersection measurement network while cooperating with the robotic arm for high-precision and fast dynamic measurement of on-orbit structural components. Figure 3(b) shows the mathematical simulation environment of the system, the intersection angle between every two cameras is 90°.

Fig. 3. Simulation environment. (a) Simulated view of OMDPS on-orbit working scenario with four cameras imaging the antenna; (b) Mathematical simulation environment.

Download Full Size | PDF

The imaging process of stars, scale bar targets, scale bar lengths and spatial target array is simulated. The imaging simulation results are obtained, as presented in Fig. 4.

Fig. 4. Imaging simulation results. (a) Cam3 image; (b) Cam4 image; (c) Cam1 image; (d) Cam2 image.

Download Full Size | PDF

4.1.2 Analysis of calibration results

In the simulation, the standard deviations of the observation error of the star image points and the scale bar target image points are set to 0.4 µm (1/10 pixel) and 0.2 µm (1/20 pixel), respectively. Defining σ_s = 0.4 µm, σ_p = 0.2 µm, and σ_l = 0.2 mm to generate the weight matrix of Eq. (13). All the parameters of the OMDPS are calibrated by the above method. To improve the calibration accuracy of the system parameters, average filtering is adopted to obtain the optimal estimated value of the parameters calibrated multiple times within a certain period.

Since there are excessive parameters involved in the optimization, only the relationship between the absolute error of the optimal estimated camera position parameters and the number of calibration times is presented in Fig. 5:

Fig. 5. The relationship between the absolute error of the best estimated camera position parameters and the number of calibration times. (a) Camera position parameter-X direction; (b) Camera position parameter-Y direction; (c) Camera position parameter-Z direction.

Download Full Size | PDF

The above results indicated that the calibration results could be stable when the calibration time reaches 50. The RMSE of star image points and scale bar target image points obtained by calibrations are 0.39 µm and 0.18 µm. The prior and posterior standard deviations of image points observation error exhibit good consistency. The absolute error of the average value of all parameters calibrated 50 times is listed in Table 2:

Table 2. The absolute error of the average value of all parameters calibrated 50 times

View Table | View all tables in this article

As depicted in the above table, this method successfully calibrates all parameters of the OMDPS. The bundle adjustment method based on star and scale bar constraints can obtain accurate and reliable calibration results.

4.1.3 3D reconstruction error analysis

Using the optimal parameters of the OMDPS obtained from the bundle adjustment, the spatial target array is reconstructed by forward intersection, and the measurement accuracy of the system is evaluated. A random error conforming to a normal distribution with a standard deviation of 0.2 µm (1/20 pixel) is introduced into the spatial target array image point coordinates. Figure 6 presents the absolute value of absolute error spatial distribution of spatial target array coordinates with single reconstruction.

Fig. 6. The absolute value of absolute error spatial distribution of spatial target array coordinates with single reconstruction. (a) X direction; (b) Y direction; (c) Z direction.

Download Full Size | PDF

As depicted in Fig. 6 and Fig. 7, all target coordinate absolute errors are in the range of ±0.1 mm in X, Y, and Z directions in the single reconstruction of spatial target array with the optimal calibration parameters.

Fig. 7. Absolute error distribution of spatial target array in three directions with single reconstruction. (a) X direction; (b) Y direction; (c) Z direction.

Download Full Size | PDF

The spatial target array is reconstructed 500 times to verify the repeatability accuracy of the system. The absolute value of the limit error spatial distribution of each point in the spatial target array is presented in Fig. 8. Figure 9 depicts the absolute error distribution in X, Y, and Z directions of a target during 500 reconstruction times.

Fig. 8. The absolute value of limit error spatial distribution of each point in the spatial target array during 500 reconstruction times. (a) X direction; (b) Y direction; (c) Z direction.

Download Full Size | PDF

Fig. 9. The absolute error distribution in three directions of a target during 500 reconstruction times. (a) X direction; (b) Y direction; (c) Z direction.

Download Full Size | PDF

As depicted in Fig. 8 and Fig. 9, the limit error and the absolute error of each point in the spatial target array are in the range of ±0.1 mm in X, Y, and Z directions during 500 reconstruction times. When the intersection angle of every two cameras reaches 90°, the absolute error of measurement for the spatial point array can be less than 0.1 mm in X, Y, and Z directions when the measurement volume is 4 m (length) × 2 m (height) × 4 m (depth).

Under the same standard deviation of the image point error of stars and scale bar target points to be 1/10 pixel and 1/20 pixel, the result of single reconstruction accuracy is shown in Table 3 when the intersection angle between every two cameras is reduced to 10° (Fig. 10).

Fig. 10. Mathematical simulation environment.

Download Full Size | PDF

Table 3. Single reconstruction accuracy

View Table | View all tables in this article

As depicted in the table, when the intersection angle between every two cameras is 10°, the RMSE in the depth direction increases to 0.1999 mm. This simulated intersection angle is the same as the system applied in the actual data ground-based experiment.

4.2 On-site experiment and results

An experimental system is designed to verify the reliability of OMDPS (Fig. 11(a)). The system employs four industrial cameras (MV-CH120-10UM/UC, HIKROBOT, Hangzhou, China) with four lenses (12FA2524-25MP, Phenix Optics, Jiangxi, China). The intersection angle between every two cameras is about 10°, and the camera baseline distance is 500 mm. Illumination is provided by four flash units (Lux Junior, God ox, Shenzhen, China), and a single camera system with flash is illustrated in Fig. 11(b). The Invar alloy scale bar shown in Fig. 12(a) is employed in the experiment. The distance between the target pairs on the scale bar is 513 mm, and the diameter of the sphere retro-reflective target is 3 mm (Fig. 12(b)).

Fig. 11. OMDPS for the actual data ground-based experiment. (a) OMDPS; (b) A single camera system with flash.

Download Full Size | PDF

Fig. 12. Invar alloy scale bar. (a) Scale bar; (b) Sphere retro-reflective target.

Download Full Size | PDF

The OMDPS is employed for outdoor experiments (Fig. 13(a)). The longitude and latitude of the experimental site is 116.2661354° and 40.1770964°. The experiment began at 22 pm. The Pegasus, Andromeda, and Cygnus nebula are observed. Referring to the calculation method of the optical system parameters of the APS CMOS star sensor [27], we calculated and selected the camera and lens needed for the OMDPS. Theoretically, the average number of stars in the field of view of the camera is 35. The star sensitive magnitude is 5 VT. The probability of OMDPS successful calibration is 100%. In the outdoor measurement, the number of stars in the field of view is about 30. The camera can be sensitive to a star magnitude of 5.2 VT. The star image coordinate and star coordinate in the celestial coordinate system can be successfully extracted and identified.

Fig. 13. On-site experimental scene. (a) Whole system; (b) Star observation test; (c) Calibrating scene; (d) Measuring scene.

Download Full Size | PDF

Figure 13(b) presents the on-site experimental scene. During the experiment, four cameras face towards the zenith, and the black shade-cloth is used to block the environment light around the system to ensure better imaging of stars (Fig. 13(c)). The system platform is rotated at a constant speed. The images of stars and four scale bars are taken synchronously. In this way, the change of spacecraft attitude can be simulated and the number of star constraints can increase. The measurement process involves star map recognition in multiple different sky areas. A total of 300 images at consecutive moments are recorded. Lastly, to verify the measurement performance of the system, the OMDPS and the V-Star system are used to perform 3D measurements of the spatial target array in the volume of 1.5 m × 1 m × 2.6 m (Fig. 13(d)). The image coordinates of stars, scale bar targets and spatial target array in every image are determined by computing the grey value centroid of the pixels in each bright spot region. Taking the measurement results of the V-Star system as the true value of the spatial target coordinates, the measurement error of OMDPS is analyzed.

4.2.1 OMDPS full-parameter calibration results

In the experiment, camera 1 is selected as the reference camera, and the results of the optimal estimation of all parameters calibrated by the system are listed in Table 4.

Table 4. The best estimation of OMDPS parameters

View Table | View all tables in this article

The image measurement error of star image points and scale bar target image points, and the length measurement error of scale bars are shown in Fig. 14 and Table 5:

Fig. 14. OMDPS image point measurement error and length measurement error distribution histogram. (a) Star image points measurement error; (b) Scale bar target image points measurement error; (c) Scale bars length measurement error.

Download Full Size | PDF

Table 5. Image point error and length error distribution statistics

View Table | View all tables in this article

The above results indicated that the image measurement RMSE of the star point is 1/8 pixel (1 σ), the image measurement RMSE of the scale bar target point is 1/17 pixel (1 σ), the length measurement RMSE is 0.0431 mm (1 σ). Using the mentioned method, all parameters of the OMDPS can be accurately calibrated.

4.2.2 Comparison of measurement performance with VS

A comparative experiment is designed to verify the measurement accuracy of OMDPS. Twelve coded targets, 321 retro-reflective targets and 1 cross directional target are arranged on the wall (Fig. 13(d)). The spatial target array is measured by VS and OMDPS, respectively. Each system is used to perform 500 measurements.

Table 6 shows the RMSE and maximum error of the measurement results of the VS.

Table 6. V-Star system coordinate measurement error

View Table | View all tables in this article

Taking the VS measurement as the true value of the spatial target coordinates, Table 7 lists the measurement error of the spatial target array coordinate measured by OMDPS. The histogram of the OMDPS measurement error distribution is illustrated in Fig. 15. The OMDPS measurement error spatial distribution is shown in Fig. 16.

Fig. 15. OMDPS measurement error distribution histogram. (a) X direction; (b) Y direction; (c) Z direction.

Download Full Size | PDF

Fig. 16. OMDPS measurement error space distribution. (a) Out-of-plane measurement error distribution; (b)In-plane measurement error distribution.

Download Full Size | PDF

Table 7. The spatial point array coordinate measurement error of OMDPS

View Table | View all tables in this article

As depicted in the above tables and figures, taking the VS measurement as the true value of spatial target coordinates, the OMDPS measurement RMSE of spatial targets in Y, Z, and X directions is 0.0428 mm, 0.0538 mm, and 0.1514 mm. It is nearly consistent with the simulation accuracy when the intersection angle between every two cameras is reduced to 10°, which further verifies the reliability of the multiple data bundle adjustment model proposed in this study.

5. Conclusion

An all-parameter calibration method of OMDPS is proposed using star and scale bar constraints. Firstly, this method uses the multi-camera relative position model to solve the reference camera position unconstrained problem in the full-parameter calibration model of the multi-camera system. Subsequently, the ill-conditioned Jacobian matrix and the reduction of adjustment accuracy in the multiple-data bundle adjustment are solved by introducing the two-norm matrix and the weight matrix. Finally, a multiple-data fusion bundle adjustment algorithm is used to optimize all parameters of the OMDPS.

In the experiment part, the proposed calibration method is verified by building a simulation system. The results of the simulation indicated that the RMSE of the measured spatial target coordinates in X, Y, and Z directions are 0.0536 mm, 0.0779 mm, and 0.1999 mm at the intersection angle of every two cameras is 10°. At the intersection angle of 90°, the limit error of the measured spatial target coordinates in X, Y, and Z directions are in the range of ±0.1 mm. The actual data ground-based OMDPS calibration and the spatial target array measurement are conducted by designing and building a prototype system. The intersection angle of every two cameras is about 10° limited by the experimental conditions. Taking the measurement of VS as the true value of spatial target coordinates, the OMDPS measurement RMSE of the measured spatial target coordinates are 0.0428 mm, 0.0538 mm, and 0.1514 mm in Y, Z, and X directions.

As revealed by the experimental results, the method proposed in this study can effectively solve the problem of insufficient KSRI in orbit to facilitate camera calibration and orientation. This method can assist the OMDPS to conduct the on-orbit calibration and orientation quickly and accurately. The global optimal calibration parameters can be used for high-precision on-orbit PG measurement tasks with multi-camera networks. Furthermore, it takes on critical significance in promoting photogrammetry technology to apply to aerospace tasks (e.g., on-orbit assembly, maintenance, and large-scale structure measurement).

At present, we adopt a small structure to ensure the portability and stability of the whole system. However, the camera baseline is short under this small structure, which leads to the small intersection angle (10°) between every two cameras of the current system. The measurement accuracy in out-of-plane direction is relatively low. According to the simulation results, when the camera intersection angle is close to 90°, the measurement accuracy in the out-of-plane direction can be further improved. The camera used in the current OMDPS has a small viewing angle (35°), which narrows the effective measurement range of the system.

In future research, we will further improve the measurement accuracy in the out-of-plane direction of the system by increasing the camera intersection angle. In order to further expand the effective measurement range of the system, it is necessary to carry out more in-depth research on the imaging model and the on-orbit calibration method of large field-of-view photogrammetry camera.

Funding

National Natural Science Foundation of China (51175047, 51475046); Scientific Research Project of Beijing Educational Committee (KM201511232020).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but maybe obtained from the authors upon reasonable request.

References

1. M. Rognant, C. Cumer, J. M. Biannic, M. A. Roa, A. Verhaeghe, and V. Bissonnette, “Autonomous assembly of large structures in space: a technology review,” in Proceedings of EUCASS 2019 (2019).

2. M. M. Wang, J. J. Luo, J. P. Yuan, J. W. Wang, and C. Liu, “In-orbit assembly technology: Review,” Acta Aeronautics and Astronautics 42(1), 47–61 (2021).

3. Z. A. Cheng, X. B. Hou, X. H. Zhang, L. Zhou, J. F. Guo, and C. L. Song, “In-orbit assembly mission for the Space Solar Power Station,” Acta Astronaut. 129, 299–308 (2016). [CrossRef]

4. S. B. Han, M. L. Dong, P. Sun, B. X. Yan, and J. Wang, “Photogrammetric network orientation of remote sensing satellite outer surface,” Chinese Journal of Scientific Instrument. 41(04), 200–207 (2020).

5. G. P. Zhang, H. Zhao, Y. H. Chen, G. D. Zhang, Z. Zhang, J. W. Peng, Z. X. Zhao, and A. Q. Yan, “Optimization thermal design method for space cameras based on thermo-optical analysis and Taguchi method,” Opt. Eng. 59(07), 075101 (2020). [CrossRef]

6. J. A. Parian, A. Cozzani, M. Appolloni, and G. Casarosa, “Thermal/vacuum measurements of the Herschel space telescope by close-range photogrammetry,” Proc. SPIE10566, International Conference on Space Optics—ICSO 2008, 105662 K (2017).

7. J. A. Parian, A. Cozzani, M. Appolloni, and G. Casarosa, “New developments with design and construction of a thermal/vacuum qualified photogrammetric system,” ISPRS J. Photogramm. 64(3), 317–327 (2009). [CrossRef]

8. D. Doyle, G. Pilbratt, and J. Tauber, “The Herschel and Planck Space Telescopes,” in Proceedings of the IEEE (IEEE, 2009), 97(8), pp. 1403–1411.

9. J. Johnston, E. Cofie, J. Hylan, R. Ohl, M. Nowak, D. McGuffey, J. Pontius, and E. Johnson, “Cryogenic thermal distortion performance characterization for the JWST ISIM Structure,” Proc. SPIE 8150, 1–9 (2011). [CrossRef]

10. B. J. Bos, D. A. Kubalak, S. R. Antonille, R. G. Ohl, J. G. Hagopian, P. S. Davila, J. Sullivan, M. Sánchez, D. Sabatke, R. A. Woodruff, M. T. Plate, C. Evans, V. Isbrucker, S. Somerstein, M. Wells, and S. Ronayette, “Cryogenic pupil alignment test architecture for the James Webb Space Telescope integrated science instrument module,” Proc. SPIE 7010, C1–C12 (2008). [CrossRef]

11. M. D. Nowak, P. E. Cleveland, E. Cofie, et al., “Cryogenic performance of a high precision photogrammetry system for verification of the James Webb Space Telescope integrated science instrument module and associated ground support equipment structural alignment requirements,” Proc. SPIE 7793, A1–A9 (2010). [CrossRef]

12. J. G. Hagopian, R. Ohl, B. Bos, P. Davila, W. Eichhorn, J. Hylan, M. Hill, M. Nowak, B. Pasquale, H. Sampler, M. Wilson, B. Gallagher, J. Hardaway, J. Sullivan, P. Young, T. Keepers, and R. Quigley, “Optical alignment and test of the James Webb Space Telescope integrated science instrument module,” in Proceedings of IEEE conference on Aerospace Conference (IEEE, 2007), Big Sky, MT, USA, pp. 1–13.

13. J. A. Connelly, B. J. Bos, P. S. Davila, et al., “Optical metrology and alignment of the James Webb Space Telescope integrated science instrument module,” Proc. SPIE 7068, 1–10 (2008). [CrossRef]

14. J. Johnston, E. Cofie, J. Hylan, R. Ohl, M. Nowak, D. McGuffey, J. Pontius, and E. Johnson, “Cryogenic thermal distortion performance characterization for the JWST ISIM structure,” Proc. SPIE 8150, Cryogenic Optical Systems and Instruments XIII, 815009 (2011).

15. T. L. Whitman, C. Wells, J. B. Hadaway, J. S. Knight, and S. Lunt, “Alignment test results of the JWST Pathfinder Telescope mirrors in the cryogenic environment,” Proc. SPIE 9904, Space Telescopes and Instrumentation 2016: Optical, Infrared, and Millimeter Wave, 990449 (2016).

16. J. B. Hadaway, C. Wells, G. Olczak, M. Waldman, T. Whitman, J. Cosentino, M. Connolly, D. Chaney, and R. Telfer, “Performance of the primary mirror center-of-curvature optical metrology system during cryogenic testing of the JWST Pathfinder telescope,” Proc. SPIE 9904, Space Telescopes and Instrumentation 2016: Optical, Infrared, and Millimeter Wave, 99044E (2016).

17. S. R. Lunt, C. Wells, D. B. Rhodes, and A. DiAntonio, “Use of close-range photogrammetry in James Webb Space Telescope alignment testing under cryogenic conditions,” J. Astron. Telesc. Instrum. Syst. 6(01), 018005 (2020). [CrossRef]

18. P. S. Zhang, S. P. Jiang, L. H Yang, and B. L. Zhang, “Development of distortion measurement system for large deployable antenna via photogrammetry in vacuum and cryogenic environment,” Proc. SPIE 10621, 2017 International Conference on Optical Instruments and Technology: Optoelectronic Measurement Technology and Systems, 106210D (2018).

19. C. A. Sapp, J. L. Dragg, M. W. Snyder, M. T. Gaunce, and J. E. Decker, Photogrammetric assessment of the Hubble space telescope solar arrays during the second servicing mission (National Aeronautics and Space Administration, Lyndon B. Johnson Space Center, 1998).

20. M. Gilbert, S. Welch, R. Pappa, M. Demeo, M. Gilbert, S. Welch, R. Pappa, and M. Demeo, “STS-74/MIR photogrammetric appendage structural dynamics experiment preliminary data analysis,” in Proceedings of 38th Structures, Structural Dynamics, and Materials Conference, AIAA1168 (1997).

21. M. Gilbert and S. Welch, “STS-74/MIR photogrammetric appendage structural dynamics experiment,” in Proceedings of 37th Structure, Structural Dynamics and Materials Conference, AIAA1493 (1996).

22. T. W. Jones, D. Liddle, J. Banik, and M. Shortis, “On-orbit Photogrammetry Analysis of the Roll-Out Solar Array (ROSA),” in Proceedings of AIAA SCITECH 2022 Forum, 1624 (2022).

23. S. L. Rickman, “Introduction to on-orbit thermal environments,” in Proceedings of Thermal and Fluids Analysis Workshop, (2014).

24. J. Wang, D. G. Jin, C. Fan, Y. Q. Bi, X. M. Su, G. Q. Liu, and Z. H. Xiang, “Predicting the on-orbit thermally induced vibration through the integrated numerical and experimental approach,” Acta Astronaut. 192, 341–350 (2022). [CrossRef]

25. T. Sun, F. Xing, and Z. You, “Optical system error analysis and calibration method of high-accuracy star trackers,” Sensors 13(4), 4598–4623 (2013). [CrossRef]

26. P. Sun, N. G. Lu, M. L. Dong, B. X. Yan, and J. Wang, “Simultaneous all-parameters calibration and assessment of a stereo camera pair using a scale bar,” Sensors 18(11), 3964 (2018). [CrossRef]

27. Y. Dong, F. Xing, and Z. You, “Determination of the optical system parameters for a CMOS APS based star sensor,” Journal of Astronautics (China) 25(6), 663–668 (2004).

Camera parameters		Cam1	Cam2	Cam3	Cam4
CEP	T_x (mm)	−3500	3500	−3500	3500
	T_y (mm)	−3500	−3500	3500	3500
	T_z (mm)	4000	4000	4000	4000
	Φ (rad)	−0.7854	0.7854	−0.7854	0.7854
	ϖ (rad)	0.6109	0.6109	−0.6109	−0.6109
	κ (rad)	0	0	0	0
CIP	f (mm)	24.7745	24.8617	24.8025	24.8221
	x_p (mm)	−0.0962	−0.1541	−0.1104	−0.0926
	y_p (mm)	−0.0570	0.0569	0.0098	−0.1180
LDP	K₁	8.05 × 10⁻⁵	7.72 × 10⁻⁵	9.30 × 10⁻⁵	6.12 × 10⁻⁵
	K₂	−3.90 × 10⁻⁷	−4.86 × 10⁻⁷	−7.29 × 10⁻⁷	−1.62 × 10⁻⁷
	K₃	−1.92 × 10⁻⁹	−8.25 × 10⁻¹⁰	5.85 × 10⁻¹⁰	−3.49 × 10⁻⁹
	P₁	5.46 × 10⁻⁵	1.62 × 10⁻⁵	4.37 × 10⁻⁵	2.76 × 10⁻⁵
	P₂	−1.61 × 10⁻⁵	−1.80 × 10⁻⁶	−8.57 × 10⁻⁶	1.29 × 10⁻⁵
	b₁	−1.07 × 10⁻⁴	7.39 × 10⁻⁵	5.32 × 10⁻⁵	−4.20 × 10⁻⁵
	b₂	1.41 × 10⁻⁶	−2.28 × 10⁻⁸	4.94 × 10⁻⁷	−9.14 × 10⁻⁷

Camera parameters		Cam1	Cam2	Cam3	Cam4
CEP	T_x (mm)	0	7.8 × 10⁻³	3.3 × 10⁻³	8.8 × 10⁻³
	T_y (mm)	0	2.2 × 10⁻³	1.5 × 10⁻³	2.9 × 10⁻³
	T_z (mm)	0	−4.8 × 10⁻³	−4.0 × 10⁻³	−2.9 × 10⁻³
	Φ (rad)	2.86 × 10⁻⁷	1.76 × 10⁻⁷	1.46 × 10⁻⁷	−6.19 × 10⁻⁷
	ϖ (rad)	−9.86 × 10⁻⁶	3.77 × 10⁻⁶	2.94 × 10⁻⁶	−1.89 × 10⁻⁶
	κ (rad)	−1.64 × 10⁻⁵	6.62 × 10⁻⁶	5.14 × 10⁻⁷	−9.39 × 10⁻⁷
CIP	f (mm)	−1.0 × 10⁻⁴	−3.14 × 10⁻⁵	−2.26 × 10⁻⁵	1.0 × 10⁻⁴
	x_p (mm)	3.2 × 10⁻⁴	−1.0 × 10⁻⁴	−3.56 × 10⁻⁶	1.31 × 10⁻⁵
	y_p (mm)	−2.0 × 10⁻⁴	7.33 × 10⁻⁵	6.23 × 10⁻⁵	−3.22 × 10⁻⁵
LDP	K₁	−2.98 × 10⁻⁷	−3.72 × 10⁻⁸	−2.44 × 10⁻⁸	2.41 × 10⁻⁷
	K₂	3.61 × 10⁻⁹	4.53 × 10⁻¹¹	1.89 × 10⁻¹⁰	−2.69 × 10⁻⁹
	K₃	−1.47 × 10⁻¹¹	1.35 × 10⁻¹²	3.56 × 10⁻¹³	9.30 × 10⁻¹²
	P₁	−2.22 × 10⁻⁷	6.81 × 10⁻⁸	2.25 × 10⁻⁸	−4.61 × 10⁻⁸
	P₂	1.51 × 10⁻⁷	−5.11 × 10⁻⁸	−4.23 × 10⁻⁸	6.47 × 10⁻⁸
	b₁	2.58 × 10⁻⁶	−4.17 × 10⁻⁸	−1.27 × 10⁻⁶	9.79 × 10⁻⁷
	b₂	1.41 × 10⁻⁶	−2.28 × 10⁻⁸	4.94 × 10⁻⁷	−9.14 × 10⁻⁷

Spatial target error	Average error (mm)	RMSE (mm)
Spatial target error	Average error (mm)	1 σ	3 σ
Z-direction	−0.0014	0.1999	0.5998
Y-direction	0.0154	0.0779	0.2339
X-direction	6.37 × 10⁻⁵	0.0536	0.1608

Camera parameters		Cam1	Cam2	Cam3	Cam4
CEP	T_x (mm)	0	473.2800	19.1593	484.6796
	T_y (mm)	0	−16.4989	429.5052	401.1768
	T_z (mm)	0	−64.4422	−25.9382	−88.6560
	Φ (rad)	−7.87 × 10⁻¹⁹	−0.0328	−0.0184	−0.0386
	ϖ (rad)	−1.72 × 10⁻¹⁸	0.0179	−0.1769	−0.1672
	κ (rad)	1.86 × 10⁻¹⁸	0.1998	−0.0003	0.2148
CIP	f (mm)	24.7552	24.8310	24.7783	24.7843
	x_p (mm)	−0.0954	−0.1356	−0.0773	−0.0967
	y_p (mm)	−0.0740	0.0404	0.0159	−0.0505
LDP	K₁	5.07 × 10⁻⁵	5.32 × 10⁻⁵	7.99 × 10⁻⁵	9.63 × 10⁻⁵
	K₂	3.43 × 10⁻⁷	5.16 × 10⁻⁷	−6.15 × 10⁻⁷	−1.80 × 10⁻⁶
	K₃	−8.39 × 10⁻⁹	−1.57 × 10⁻⁸	−8.59 × 10⁻¹⁰	1.93 × 10⁻⁸
	P₁	4.87 × 10⁻⁵	−8.05 × 10⁻⁶	2.10 × 10⁻⁵	4.36 × 10⁻⁶
	P₂	−1.37 × 10⁻⁶	−2.21 × 10⁻⁶	−1.02 × 10⁻⁵	−1.42 × 10⁻⁵
	b₁	−0.0001	8.52 × 10⁻⁵	−0.00012	−7.51 × 10⁻⁵
	b₂	−1.81 × 10⁻⁵	−1.44 × 10⁻⁵	2.66 × 10⁻⁵	0.0002

Data type	Error type	Average error	RMSE
Data type	Error type	Average error	1 σ	3 σ
Star image point	Image plane error (µm)	−0.0013	0.4251	1.2752
Star image point	Pixel error (pixel)	1/2653	1/8	1/3
Scale bar target image point	Image plane error (µm)	0.0008	0.1989	0.5967
Scale bar target image point	Pixel error (pixel)	1/4312	1/17	1/6
Scale bar length	Length error (mm)	3.61 × 10⁻⁶	0.0431	0.1293

All-parameter calibration method of the on-orbit multi-view dynamic photogrammetry system

Abstract

1. Introduction

2. Mathematical model

2.1 Camera imaging model

2.2 Star imaging model

2.3 Scale bar target imaging model

2.4 Scale bar length model

3. Full-parameter bundle adjustment algorithm

4. Experiment and discussion

4.1 Simulation experiments and results

4.1.1 Simulation system and scene configuration

4.1.2 Analysis of calibration results

4.1.3 3D reconstruction error analysis

4.2 On-site experiment and results

4.2.1 OMDPS full-parameter calibration results

4.2.2 Comparison of measurement performance with VS

5. Conclusion

Funding

Disclosures

Data availability

References

Data availability

Cited By

Figures (16)

Tables (7)

Equations (14)

Optics Express

Spatial target error	Maximum error (mm)	RMSE (mm)
X-direction	0.008	0.006
Y-direction	0.004	0.003
Z-direction	0.004	0.003