1. Introduction

The monocentric objective lens has the advantages that the conventional transmitted focal-plane imaging objective lens cannot achieve in the large field of view, large relative aperture, and high imaging resolution [1–3]. It also has the characteristics of small size, light weight, and strict symmetry structure. Additionally, all the monocentric objective lenses are absolutely monocentric, and high symmetry removes most geometric aberrations. Therefore, it is not necessary to correct external aberrations like coma, astigmatism, and lateral chromatic aberration when optimizing the imaging quality and correcting the positive aberration of the monocentric objective imaging system. This greatly reduces the time and expense of aberration correction and is helpful for enhancing the resolution and clarity of images [4]. However, the imaging focus surface is a curved hemispherical surface due to the unique optical design of the monocentric objective lens, making it challenging to correctly couple with the traditional array image sensor. It will result in severe defocus and considerable imaging target dispersion and blur at the array sensor's edge, which previously restricted the use of monocentric objective lenses.

A type of discrete, passive, rigid optical fiber image transmission device known as an fiber optic plate (FOP) is created by secondary drawing, melting, and cutting millions of optical fiber monofilaments. FOPs have found widespread usage in the domains of national defense, scientific research, aerospace, and medicine thanks to their optical properties of high numerical aperture, low optical loss rate, zero optical thickness, high resolution, and high transmittance. The most significant advantage of the FOP is that its end face can be prepared into the shape of an optical surface, which not only allows it to function as a means of transferring images between different optical surfaces but also significantly reduces the size of the entire optical structure, improving the stability of the system.

By turning the incident-exit end face of the FOP into a concave spheric-plane and coupling it directly with the hemispherical focal plane of the monocentric objective lens and the surface of the image sensor as a relay image transmission device, the issue that the focal plane of the monocentric objective lens cannot be coupled with the sensitive surface of the sensor can be solved perfectly. In order to produce a large-field image with hundreds of millions or even billions of pixels [5], the several FOPs coupled large-field array space-class image sensors are spliced through a particular spatial arrangement that takes into account the large field of view and high resolution imaging [4,6–9].

The introduction of FOPs will, however, inevitably cause a decline in the imaging quality of monocentric cameras, such as lower transmittance, imaging information discontinuity, an increase in imaging target centroid locating error [10], and so on since they are discrete imaging devices. Moreover, during the production of FOPs, intrinsic flaws like dark filaments, broken filaments, or even block damaged filaments would unavoidably happen, leading to uneven or even non-responsive pixel responses of image sensors.

Currently, the technical specification of Photonis in France, which clearly states: ‘A spot is defined as any area (chicken wire and/or dead fibers) where the relative transmission is less than 70%,’ serves as the international definition of inherent defects of optical fiber image transmission devices. Research on optical fiber image transmission devices and their uses has been done in-depth by Schott Company in Germany and Hongsheng Optoelectronics in China. They produce a variety of high resolution, low monofilament diameter, large section FOPs while continuously improving the production process and strictly regulating the proportion of defective monofilament. It is vital to identify and address the inherent flaws in each FOP during the practical application process since, despite substantial advancements in the fabrication technology of FOPs, these flaws cannot entirely be eradicated.

Image segmentation and edge detection are the two main techniques used to find defects in FOPs. Image segmentation is the process of separating the defect area from the background of the image in order to achieve the goal of immediately labeling the faulty area. Edge detection should first establish the perimeter of the image defect area before using morphological processing to designate the size and position of the defect. Wu et al [11]. achieved the FOP shadow defect edge detection in 2008 utilizing the canny operator. Two years later, they [12] realized the adaptive threshold determination, integrated the Otsu algorithm into the canny operator, and attained a nearly perfect detection result for the defective edge of the FOP. The image segmentation approach was later discovered by the researchers to be more practical and effective for the fault identification of FOPs. The most popular global threshold segmentation algorithm, the Otsu algorithm, can provide good defect detection effects, but it can also result in misleading defect detections like noise and dark filaments [13]. In 2017, Yang et al [14]. introduced an improved adaptive watershed segmentation technique that can successfully control the over-segmentation of FOP flaws. A defect detection technique based on the enhanced Fuzzy C-means clustering algorithm was put out by Zhang et al [13]. in 2018. This approach can prevent detection errors brought on by fictitious flaws, but it converges quickly locally and operates slowly with big data sets. Because of this, it is not appropriate for large-section FOPs having a specific number of unevenly distributed flaws. Additionally, in recent years, researchers have used machine vision technology to detect winding defects in fiber coils [15,16] and end faces [17], but this technology is more suited for real-time, linear array detection or local range detection and is not appropriate for detecting global defects in large array FOPs.

The image surface of a monocentric camera is made up of a huge array of space-class CMOS sensors, with a maximum pixel count of 100 million. Using the global threshold segmentation approach to find defects in FOPs is obviously more practical and effective. The FOP also contains a large number of dark filaments with low transmittance, which can nonetheless transfer energy and reveal the target's radiation properties. If the black filaments are mistakenly identified as flaws, the processed image will have extra pitting areas, which cannot be used. It makes more sense, therefore, to think of these black filaments as the non-uniformity phenomenon of camera image. In this paper, a global defect detection method based on manual threshold segmentation of saturated frames is proposed in light of the defect characteristics of FOPs. This technique can prevent the unintentional detection of noise and dark filaments. The proposed method's efficacy and accuracy are confirmed when compared to the traditional Otsu algorithm. Additionally, various FOP-related picture inhomogeneity events are described in detail. Experiments are used to determine the relative response coefficients of each pixel, and response non-uniformity is corrected at the pixel level. In the realm of space-based surveillance, the study in this paper expedites the use of monocentric cameras based on fiber relay imaging, and the research methodology is also transferable to the large array fiber relay image transmission system.

2. Defect pixel detection of monocentric camera

The term “defect pixels” in the context of classic optical cameras primarily refers to those pixels, also known as “poor points,” such as bright spots, dark spots, and defective rows or columns, which are unable to respond correctly during the manufacture and packing of image sensors. In addition to the poor pixels brought on by the image sensor's inherent flaws, monocentric optical cameras also have bad pixels brought on by the FOP's inherent flaws. The following are the primary categories of FOPs’ inherent flaws [18,19] (see Fig. 1):

 

Fig. 1. Types of inherent defects of FOPs: (a) chicken filaments (b) dead spots (c) grid (d) frame (e) pollutants.

Download Full Size | PDF

The aforementioned intrinsic flaws will cause the final output signal value of the corresponding pixel to be much lower than that of the pixel without flaws in the condition of uniform light incidence. The optical fiber transmittance will drop as a result of the FOP's flaws, but the rate at which it will decrease will vary depending on the type of flaw. Hence, identifying these damaged pixels and noting their precise locations is crucial for following image calibration and enhancing cameras’ overall imaging quality. The most used image segmentation technique is Otsu algorithm, and it is typically used to distinguish the inherent flaws of FOPs from the regular image. The Otsu algorithm, which determines the threshold for image binarization segmentation, is based on the maximization of inter-class variance. According to the global threshold, the algorithm thinks that image pixels can be split into background and target sections. In order to achieve the greatest degree of differentiation between the two types of pixels, the best threshold is computed to separate the two types of pixels. Although the Otsu algorithm is straightforward and effective, it also has the following drawbacks: (1) It is simple to create local noise misdetection since the algorithm is particularly sensitive to noise. (2) The inter-class variance function may exhibit double or numerous peaks when the target and background have too many pixels in common, which lowers the algorithm's detection accuracy.

The geometric optics principle states that when the camera's field of view Angle $\omega$ increases, the side illumination will result in an energy attenuation proportionate to ${\cos ^4}\omega$, creating an image that is bright in the center and dark on the edges, resulting in a dark Angle. The dark corner area with a low output signal value will be treated as a defect at this moment since the Otsu method is being used for segmentation, which causes a partial loss of picture data. A qualified FOP's defect ratio should also not be higher than 0.5% [18,20]. Low accuracy in the segmentation of FOP defects is caused by the Otsu defect segmentation method for global images’ ease in displaying double-peak or multi-peak inter-class variance functions. The FOP will simultaneously produce low transmittance dark fiber filaments that are scattered at random. While having a lower transmittance than optical fiber monofilaments with normal image transmission, this type of dark mercerization can nonetheless transfer images and characterize variations in target radiation brightness. The non-uniformity of the subsequent imaging response can totally remedy the filaments issue, therefore it cannot be viewed as a flaw. The Otsu algorithm, however, will mistakenly identify this filaments behavior as a flaw.

In order to avoid the interference of random noise, the erroneous identification of optical fiber dark filament, and the effects of dark Angle on the detection accuracy, this paper provides a straightforward, effective, and highly accurate FOP defect detection method. The detection method's precise operational flow is as follows:

It is important to clarify that the monocentric optical camera's image surface is made up of several image sensors and has a wide field of view. The irradiance received by the corners and edges of the picture plane will result in an attenuation proportionate to ${\cos ^4}\omega$, according to the geometric optics principle. The attenuation increases with increasing field of view. The image surface of the image sensor in the center field of view must be supersaturated in order to guarantee the overall saturation of all image sensor surfaces. This will make the faults of some FOPs in the center also saturated, resulting in the inaccuracy of defect identification. In order to avoid the inaccuracy of defect detection brought on by irradiance attenuation, it is necessary to roughly adjust the camera's lens pointing when acquiring saturated frame images so that the lens area corresponding to the center field of view of the acquired image sensor is facing the integrating sphere (see Fig. 2). This method involves changing the pointing of the camera lens for each saturated frame image component that is captured by the image sensor. Using the aforementioned technique, the binary image of the position of the FOP defect is obtained. The global DN mean value of the following photos is not computed using defective pixels.

 

Fig. 2. Schematic diagram of monocentric camera defect detection.

Download Full Size | PDF

To identify the five FOP flaws depicted in Fig. 1, we employed the defect detection method described in this paper and the Otsu algorithm, respectively. In Fig. 3, the test results are displayed. For the sake of readability, we define white as a defective pixel and black as a regular pixel. The defect detection approach in this study has higher detection accuracy, wider coverage, and a larger degree of reduction when compared to the conventional Otsu algorithm, as can be shown by comparing the detection results of the two detection methods.

 

Fig. 3. Comparison of the defect detection results of this article's method and the Otsu algorithm: (a) represent the defects that need to be detected; (b) represent the defect detection results of this article's method; (c) represent the defect detection results of the Otsu algorithm.

Download Full Size | PDF

3. Non-uniformity correction of imaging response for monocentric camera

Even after defect identification and dark field calibration, the output gray value of each pixel of the image captured by the camera under the condition of uniform light incidence is still inconsistent, demonstrating non-uniformity. The numerous causes of imaging heterogeneity can be loosely categorized into three groups:

The deployment of FOPs will make the pixel response heterogeneity worse. Figure 4 (a), (b), and (c) depict three different types of pixel response heterogeneity brought on by an FOP, respectively:

 

Fig. 4. Response nonuniformity phenomenon caused by the FOP: (a) (group) dark filaments (b) hexagonal spots (c) inclined stripes.

Download Full Size | PDF

Whatever the root cause of the aforementioned problem of non-uniform imaging response, it will eventually result in inconsistent response between image sensor elements. It is required to rectify the non-uniformity of the imaging response for calibrated images in order to remove the non-uniformity of imaging under uniform illumination conditions.

3.1 Linear fitting of flat field image based on least square method

It is important to calibrate the pixel level relative response relationship between the camera output gray value and entry pupil radiance prior to the non-uniformity correction of the image response. When the radiance of the camera entering the pupil is defined as L, the output DN of the image sensor pixel with coordinate $({m,n} )$ is defined as $D{N_0}({L,m,n} )$. Pixel output DN value excluding dark field image $Dark({t,m,n} )$ is defined as:

Similar to this, we can set the image sensor's pixel output DN value to $DN({L,m,n} )= [{DN({{L_1},m,n} ),DN({{L_2},m,n} ),\ldots ,DN({{L_k},m,n} )} ]$ when the entrance pupil radiance is $L = [{{L_1},{L_2}},$$\ldots ,{L_k} ]$. Assuming that A and B are positively correlated, the following fitting formula can be obtained by linear fitting based on least square method:

 

Fig. 5. The least-square method-based linear fitting graph between the image means and radiance.

Download Full Size | PDF

Figure 5 shows intuitively that the image sensor has a good linear response to radiation brightness, but it is still unknown what the relationship between the two is. First, the absolute difference between the actual image output mean and the linear fitting value under each radiation brightness level is obtained, namely the residual $\delta$. In the linear regression analysis, the residual $\delta$ obeys the normal distribution $\textrm{N}({\overline \delta ,{\sigma_\delta }^2} )$. Where, $\overline \delta $ stands for the residual mean, and ${\sigma _\delta }$ stands for the residual standard deviation. Define the standardized residual $\delta ^{\prime} = {{({\delta - \overline \delta } )} / {{\sigma _\delta }}}$, which follows the standard normal distribution $\textrm{N}({0,1} )$. We can obtain the residuals and standard residuals between the actual output mean and fitting values of the images at each brightness level (see Table 1). The standard residual has a 0.05 chance of falling outside of range $({ - 2,2} )$. With a 95% confidence level, an experimental point will be classified as abnormal and excluded from the regression linear fitting if its standardized residual is outside the interval. Table 1 demonstrates that all of the data points used in the fitting process have standardized residuals that fall within interval $({ - 2,2} )$. The correlation coefficient between the radiation brightness level and the actual image output gray mean was also obtained; its value of 99.99%, almost 1, shows a positively linear correlation between the two.

Table 1. The fitting residuals and standard residuals between image means and radiance

View Table

3.2 Non-uniformity correction of imaging response

Non-uniformity correction of imaging response refers to the fact that, after correction, the pixel output DN value of the image sensor remains constant when the camera is exposed to uniform light. The single point method, double point method, multi point method, and others are often used flat field correction techniques. If both the gray value of the picture to be calibrated and the correction coefficient of non-uniformity of the image response are known, the single point technique can calibrate the gray value of the image after non-uniformity correction. This method requires the camera to have a high linearity, and after subtracting the dark field signal, the output signal value of the camera almost has no offset, that is, the offset is nearly regarded as zero. The principle formula of single point method can be expressed as:

The pixel output DN value following correction is set as the image mean value to guarantee that the total amount of irradiance received by the corrected image plane stays constant. The following formula is established if, after correction, the output DN value of the pixel with coordinate $({m,n} )$ is $D{N_{rev}}({L,m,n} )$.

After deducting the dark field image, it is found that $\overline B - \frac{{\overline R }}{{R({m,n} )}}B({m,n} )$ approaches zero. We define the non-uniform correction coefficient of pixel imaging response as ${R_{rev}}({m,n} )$, then the following formula is valid.

Combined with formula (1) and (8), formula (7) can be rewritten as:

In formula (9), image $D{N_0}({m,n} )$ to be calibrated is a known quantity, and the value of ${R_{rev}}({m,n} )$ can be obtained by formula (8). There are typically two ways to get the dark field $Dark({t,m,n} )$ of the calibrated image.

Case 1: Use the identical shooting conditions to capture the dark field image as you did for the calibration image.

Case 2: Maintain the working temperature of the calibrated image, fit the integration time and dark current value linearly, and use the fitting coefficient to derive the calibrated image's dark current value with a known exposure duration.

The calibration image's dark field is best obtained using Case 1. To sum up, when image $D{N_0}({m,n} )$ to be calibrated is input. According to the correction formula (9), the image $D{N_{rev}}({m,n} )$ after the non-uniformity correction of imaging response can be obtained.

3.3 Results of imaging response non-uniformity correction

The CMOS installed on the monocentric optical camera is an aerospace-class image sensor, employed primarily for the detection of weak and indistinct space targets. As a result, this sensor has high sensitivity and gain, as well as the qualities of being easily saturated. After the initial test, the integrating sphere can be controlled by only one bulb being opened at once, which is sufficient to fulfill the conditions of the experiment. Open the camera visor, place the front imaging lens close to the integrating ball's light outlet, and make sure the camera's entire field of vision can receive uniform light incidence from the ball (see Fig. 6). It is discovered that the camera output DN value is approximately one-tenth of the saturated DN value by providing instructions to regulate the camera integration time as unit integration time (${t_{unit}}$, 51.2 microseconds). As a result, we set the camera's integral time to be $t = [{{t_{unit}},2{t_{unit}},\ldots ,9{t_{unit}}} ]$ correspondingly. Twenty frames of pictures were measured for superposition processing under each integral time condition. It's important to note that temperature management or heat dissipation techniques should be used to maintain the image sensor's operating temperature during all integral time shooting circumstances.

 

Fig. 6. Diagram of monocentric camera imaging correction experiment.

Download Full Size | PDF

According to $L \propto E(\lambda )\times t$, it can be shown that integration time t and target irradiance $E(\lambda )$ are proportional to entrance pupil radiance L. The change in the dark current value of the picture obtained under each integration time condition is also very modest and may be disregarded because the camera integration time is on the order of unit integration time. Controlling the change of integrating time can therefore be equal to managing the change of the entry pupil's radiance when the brightness of the integrating sphere light source is fixed. The assumption underlying the equivalence relation is that the integration time is on the order of unit integration time. However, as the integration time increases, the dark current output value on the order of seconds will change significantly, negating the applicability of the aforementioned equivalent relation. Moreover, because absolute radiation calibration calls for calibrating the radiance value of the integrating sphere, this approach is only relevant to relative radiation calibration. Also, because of the incredibly little exposure duration, less random noise will be added, which might lessen the impact of noise on the calibration accuracy.

The field of view of a monocentric optical camera is composed of image sensors with multiple coupled FOPs. Figure 7 displays the grayscale value of the signal output obtained by each image sensor under the circumstance of uniform light incidence with various brightness levels.

 

Fig. 7. The DN mean values of all spliced image sensors under different irradiance conditions.

Download Full Size | PDF

Image sensors 1, 4, 5, and 8 are far from the center field of view area, hence the DN value is low. Image sensors 2, 3, 6, and 7 are in the center field of view area, indicating a high DN value. As a result, we must choose an appropriate pixel output value to minimize the non-uniformity coefficient of the corrected average pixel response, which can be achieved by using the effective pixel output mean value of the spliced image sensor [22]. Formula (3) is changed to formula (10) after setting the number of stitching image sensors to k. The mean value of effective pixel output and the fitting curve for each brightness level are displayed in Fig. 7. The fitting curve can be used to determine the average response coefficient of the image sensors after stitching.

 

Fig. 8. The flat field images before and after non-uniformity correction (IS-X stands for image sensor X): (a) Before correction; (b) After correction.

Download Full Size | PDF

The accuracy of the non-uniformity correction is evaluated using the image non-uniformity. The following formula defines image non-uniformity as the ratio of the standard deviation and mean values of the output signal values of the effective pixels of the image sensor under uniform light incoming conditions:

 

Fig. 9. The realistic scene images before and after non-uniformity correction: (a) Before correction; (b) After correction.

Download Full Size | PDF

4. Conclusion

In this study, a novel type of space-based surveillance system, a monocentric optical camera based on fiber optic relay image transmission, is used for imaging defect detection and response non-uniformity correction. First, a defect detection method of manually binarizing segmenting saturated frames is proposed based on the defect characteristics of FOPs. Results show that this method is more applicable and has a higher detection accuracy when compared to the traditional image segmentation method Otsu algorithm. In order to correct the imaging response heterogeneity, it is also necessary to mark the relative response coefficients between the pixels of flat-field images with varying levels of radiation brightness. The correction coefficients are then derived from the experimental data. The analysis of the pixel inhomogeneity before and after rectification comes last. The outcomes demonstrate that after rectification, the heterogeneity of pixel response falls from 10.01% to 0.78%. The method described in this study can reduce the monocentric camera's image surface's pitting points, remove interference from the camera's response error, and enable the camera image to respond to actual target characteristics. The study's findings can be used to improve other optical fiber relay picture transmission systems. Next, we will use the findings from this paper's research to process real star maps while also examining the target energy concentration, the probe star, and other indicators of processed images to hasten the implementation of the monocentric camera based on fiber relay imaging in the field of space-based surveillance.