Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Compact, UAV-mounted hyperspectral imaging system with automatic geometric distortion rectification

Open Access Open Access

Abstract

A highly compact hyperspectral imager with an automatic geometric rectification function is developed in this study, which can be mounted on a UAV for ultra-wide range hyperspectral imaging. For better application, the system can provide visible light image transmission and hyperspectral imaging in the real-time mode. A specific design is proposed to allow the visible light camera and hyperspectral camera to share the same telescope optical path, making the system have a high integration level with a total mass of 1.9 kilograms. Thanks to the sharing-optical-path design, the field of view (FOV), frame rate, and spatial resolution are modified the same between the visible light camera and hyperspectral camera. As a result, the geometric rectification is easily performed, and repeated rectifications are eliminated to improve the imaging efficiency. A FOV of 40 degrees in the frame direction and 26 degrees in the flight direction are realized with a focal length of 13mm, providing a large spectral range from 400 nm to 1000 nm and an excellent spectral resolution of 2.5 nm. An automatic geometric rectification workflow is presented and verified in experiments, which can improve the geometric rectification of hyperspectral images in the presence of low-quality UAV navigation data through the incorporation of frame images. Experimental results show that the relative accuracy of geometric rectification is less than 2 pixels when applying the algorithm to our system dataset.

© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Hyperspectral technology is effectively capable of capturing more detail in both spectral and spatial domain to study materials and organisms, researchers have paid a lot of efforts on the miniaturization work to make the hyperspectral imaging system more compact, as a result of which the system can be integrated in Unmanned aerial vehicle (UAV) for either scientific or commercial purposes. UAV-based hyperspectral imaging is a relatively new remote sensing technology. Compared with the traditional satellite and airborne platforms, near-earth spectroscopy from UAV-hyperspectral systems break the limitations of the resolution, signal-to-noise ratio (SNR), which could satisfy the needs of a wide range of civilian applications, including the atmosphere, ocean, agriculture, forestry, geology and minerals, environmental protection, resource investigation and so on. Due to the above advantages and its wide applications, highly compact and portable hyperspectral technology has attracted great attention from both research field and engineering area. As to specific application approaches, especially when integrated to mobile platforms such as drones, accurate geometric correction of collected images is essential, which is very important for dynamic detection and/or locating the target samples on the ground [17].

In the process of data acquisition, drones inevitably produce mechanical jitter and flight deviation. Moreover, hyperspectral scanners captures scenes commonly based on a push-broom mode (i.e., the scene coverage is achieved through multiple exposures of the detector during the platform’s motion along its trajectory). Therefore, the geometric fidelity of derived information from push-broom hyperspectral scanners is quite sensitive to the available position and orientation established through a direct geo-referencing unit onboard the UAV platform (i.e., an integrated Global Navigation Satellite System (GNSS) and Inertial Navigation System (INS). Every little change in pitch, roll and yaw during flight path is transcribed into the data, the obtained hyperspectral data, due to the influence of aircraft motion and air flow, will have large geometric distortion, which will seriously affect the image quality, sometimes even cause the image to be unable to be visually interpreted. Representative products include Specim in Finland and Headwall Nano in the United States, however, the unavoidable geometric noise problem in the work process represents the general dilemma of such a type of UAV-based hyperspectral imaging system [811].

The inherent weaker imaging geometry of push-broom hyperspectral scanners due to their multiple exposures of the linear array during the platform’s motion along its trajectory requires accurate navigation data. The commonly used methods are Ground Control Points (GCPs) and the implementation of an integrated GNSS/INS unit onboard the mapping platform. The latter is the preferred option since GNSS/INS-based direct geo-referencing can reduce or even eliminate the need for establishing GCPs, which is quite expensive and operationally impractical for precision agriculture and environmental monitoring based on UAV platform. In addition, the endurance and payload constraints are major factors that could impede the deployment of comprehensive UAV-based platform. To cope with the limited endurance and payload constraints of low-cost UAVs, therefore, the remote sensing and agricultural communities mainly rely on UAVs equipped with consumer-grade direct geo-referencing and imaging systems, which are relatively small and light weight, providing inaccurate position and orientation information compared to those equipped with survey-grade GNSS/INS units (such as LEICA ADS40) [12]. In general, the accuracy of civil inertial attitude measurement products (such as the MESE gyroscope) is only 0.1 degree, which is unable to perform pixel-level geometric rectification on hyperspectral images with resolution requirements of milliarc scale [1315].

In this study, a highly compact hyperspectral imager with automatic geometric rectification function is developed, which can be mounted on UAV for ultra-wide range hyperspectral imaging. A FOV of 40 degree in the frame direction and 26 degree in the flight direction are realized with a focal length of 13 mm, providing a large spectral range from 400 nm to 1000 nm, an excellent spectral resolution of 2.5 nm is achieved based on our design. More details will be discussed as follows in section 2, 3. An experiment is performed to validate the method of automatic geometric rectification in the section 4, geometric correction is applied to our dataset to see the performance. Experimental results show that the relative accuracy of geometric correction is less than 2 pixels when applying the algorithm to our system dataset.

2. Optical design

2.1 Overall design content

The working band of the hyperspectral imager is 400–1000 nm, and the field of view is 40 degree (width direction) 26 degree (flight direction), and the focal length is 13mm, F# 2.8. The high-frame-rate area array camera and hyperspectral camera share the same telescope optical path, making the system a highly integration level of a total mass 1.9 kilogram. The detector selection of area array camera is the same as that of the imaging spectrometer, which is convenient for electronic system integration. The pixel size of the selected detector is 5.5 µ m × 5.5 µ m, and the effective number of pixels used is 2000 (width direction) × 1200 (flight direction). In order to meet the requirements of geometric rectification of hyperspectral images and acquisition of 3D ground feature information, the field angle, frame rate and spatial resolution of area array camera are the same as those of large field hyperspectral camera.

When the flight altitude is 100 m, the corresponding ground width is 73 m, and the spatial resolution of large field imaging spectrum is 7.2 cm. The main technical indicators of the optical system are shown in Table 1.

Tables Icon

Table 1. Technical specifications of imaging spectrometer

The integrated GNSS/INS unit onboard mounted on UAV cannot meet the accuracy requirements of the direct geometric rectification of push-broom hyperspectral images in terms of positioning accuracy and sampling frequency. The configuration requirements of the survey-grade GNSS/INS units in terms of size, weight, and power consumption, exceeds the limited durability and payload limitations of civilian small drones. Figure 1 shows the pointing deviation of the spatial information of the UAV-based hyperspectral data during data collection (the ground position of the image changes due to the position and attitude changes between frames). It can be seen that there are serious geometric distortions in the use of UAVs to obtain push-broom hyperspectral data. The high-frame-rate area array detector can completely save the position and attitude information of each frame. In order to realize the high-precision geometric rectification of hyperspectral data, the frame images are used to realize the geometric rectification processing of the push-broom hyperspectral image. The high-frame-rate area array camera and the large-field imaging spectrometer share the front telescope, and use prism or the method of dividing the field of view by the color splitter will greatly reduce the system transmittance and sacrifice the load performance. Therefore, the knife-edge mirror is used to divide the field of view, and the total field of view of the system is divided into a plane field of view and a line field of view. The plane field of view is received by the area array camera, and the line field of view is used as the incident slit of the spectrometer system. The image geometry relationship between the large-field imaging spectrometer and the area array camera is shown in Fig. 2. In order to meet the geometric rectification of hyperspectral images and the acquisition of three-dimensional object information, the field of view, width, frame rate, and spatial resolution of the area array camera are the same as those of the imaging spectrometer [16].

 figure: Fig. 1.

Fig. 1. The deviation of spatial information during data collection

Download Full Size | PDF

 figure: Fig. 2.

Fig. 2. The geometric relationship between imaging spectrometer and area array camera

Download Full Size | PDF

Under the constraints of weight, size, field of view, the UAV load system also requires the system to have a high light collection capability and modulation transfer function, while minimizing the system's spectral line bending and color distortion. Considering the requirements of volume, weight, transmittance, data rate, etc., a convex spherical grating spectroscopy design scheme is adopted, and the Offner concentric structure is used as the optical structure of the spectrometer. It has a large field of view, uniform dispersion, small spectral line curvature, and small color distortion. Realize miniaturization and light weight at the same time with other characteristics. Independent research and development of triangular groove type holographic diffraction grating, while overcoming the low diffraction efficiency of traditional rectangular groove type grating, obtains the optimal system detection ability under load constraints. When the grating groove shape is triangular, the designed grating blaze angle is 1.06°. At this time, the diffraction efficiency curve of the grating is shown in Fig. 3. When the grating groove shape is rectangular, the designed groove depth is 2.6 µm and the duty ratio is 0.5. At this time, the grating diffraction efficiency curve is shown in Fig. 4.

 figure: Fig. 3.

Fig. 3. Diffraction efficiency of triangular grooved grating

Download Full Size | PDF

 figure: Fig. 4.

Fig. 4. Diffraction efficiency of rectangular groove grating

Download Full Size | PDF

2.2 Design of the telescope system

The function of the telescope system is to image the light reflected from ground objects into the slit of the spectrometer. The working wavelength range is 400 nm ∼ 1000 nm. Under the design constraints of large field of view and short focal length, the design scheme of rigid segmentation field of view is adopted to realize folding structure and reduce the system size. The area array camera and the imaging spectrometer share the telescope, which can reduce the volume and weight of the system, increase the structural rigidity of the system, and improve the optical axis stability of the spectrometer and the area array camera. The telescope adopts a transmissive structure. In order to achieve pupil matching with the spectrometer system and ensure the accuracy of geometric rectification, the telescope adopts an image-side telecentric design. The field of view is 40° x 26° and the focal length f’ is 13 mm. The system is designed to achieve the largest field of view of similar products under high integration requirements. Figure 5 is the spot diagram on the image plane of the telescope system. The Root Mean Square (RMS) radius is less than 1/2 equivalent pixel, which satisfies the imaging quality requirements.

 figure: Fig. 5.

Fig. 5. Spot diagram on the image plane of telescope

Download Full Size | PDF

2.3 Design of the spectral imaging system

The spectral resolution of the hyperspectral imager is determined by the spectral imaging system, and the spatial resolution is also closely related to the imaging quality of the spectral imaging system. Although the prism beam splitting scheme has a great advantage in transmittance, the energy utilization rate can reach 90%, but the weight of the system is limited and cannot meet the practical application of unmanned aerial vehicles. The spectral imaging system based on Offner convex grating has the smallest volume and weight, and meets the requirements of miniaturization and light weight of the load. In terms of optical design, aberration vignetting caused by diaphragm aberration is used to improve the uniformity of the image surface illuminance to achieve a large field of view, high imaging quality, and uniform image surface. The size of the incident slit of the spectrometer system is 10.4mm×0.045mm, the numerical aperture is 0.179, and the zoom ratio is 1/1. The off-axis spherical spectroscopic imaging system is improved, the collimator lens and imaging lens are replaced with aspheric surfaces, and the large-diameter correction lens is eliminated. Reduce the number of lenses, improve imaging quality and system transmittance. On the other hand, the convex grating adopts a triangular groove design with high diffraction efficiency, and the diffraction efficiency finally achieves more than 80%. The light emitted from the entrance slit is incident on the aspherical collimating lens, collimated by the aspherical collimating lens, and then incident on the working surface of the convex grating. Finally, the aspherical focusing lens is used to focus on the detector to realize the focusing of the dispersive beam. In terms of detectors, a large area array and high sensitivity CMOS detector developed by Gpixel is selected.

2.4 Complete system design of hyperspectral imager

The optical structure of the entire system of the hyperspectral imager is shown in Fig. 6. The optical transfer function curves of the hyperspectral imager at different wavelengths are shown in Fig. 7. It can be seen that at the characteristic frequency, the optical transfer function is greater than 0.68, which meets the imaging quality requirements of the hyperspectral imager. The volume of the optical system is 200 mm×120 mm×70 mm [17].

 figure: Fig. 6.

Fig. 6. Optical path of hyperspectral image

Download Full Size | PDF

 figure: Fig. 7.

Fig. 7. MTF of hyperspectral imager for different wavelengths: (a) 400 nm; (b)700 nm; (c) 1000nm

Download Full Size | PDF

3. System integration and testing

3.1 Mechanical structure design

According to the optical design results, the mechanical structure design was completed. If the structure is designed directly according to the form of the optical system, the overall structure will become slender, which is not good for rigidity. Therefore, a folding mirror must be added to the optical system to make the overall layout reasonable by folding the optical path. The overall mechanical structure model is shown in Fig. 8. The whole machine adopts a modular structure, and the telescope, spectrometer, and area array camera each form a module. Since the telescope and area array camera must be connected to the spectrometer, the spectrometer is selected as the main structure. The telescope and area array camera are fixed on the spectrometer, and the spectrometer is fixed on the airborne platform. The total quality of the system is controlled within 2kg.

 figure: Fig. 8.

Fig. 8. The mechanical structure model

Download Full Size | PDF

The mechanical characteristics of the hyperspectral system under the overall structure scheme are analyzed, and the modal analysis of the whole machine is carried out. The finite element model and boundary constraints are shown in Fig. 9, the first three-order mode shapes are shown in Fig. 10, and the first three-order natural frequencies and mode shapes are described in Table 2. The analysis results show that the first-order frequency of the system is 304.67 Hz, which has good dynamic characteristics and can meet the requirements of UAV mechanical conditions.

 figure: Fig. 9.

Fig. 9. Finite element model and boundary conditions

Download Full Size | PDF

 figure: Fig. 10.

Fig. 10. The first three natural modes of hyperspectral system

Download Full Size | PDF

Tables Icon

Table 2. The first three natural frequencies and modes

3.2 Assembly, adjustment and calibration

The centering device is used to ensure the angle deviation and the center interval deviation during the optical system installation and adjustment process. The transfer function and distortion tester of Optiks company are used to test the imaging quality of area array detector. Figure 11 shows the setup and adjustment process of the spectral imaging system. Both the collimator lens and the imaging lens are off-axis aspheric lenses; however, they can be adjusted by a common reference (standard plane lens of the Zygo interferometer) [17].

 figure: Fig. 11.

Fig. 11. Setup and adjustment process of hyperspectral imaging system

Download Full Size | PDF

Figure 12 shows the designed wavefront aberration of the spectral imaging system, which is 0.136 λ, Fig. 13 shows the actual measured wavefront aberration, which is 0.146 λ, it is closed to the designed wavefront aberration shown in Fig. 12, thereby indicating that the spectral imaging system achieves good imaging quality.

 figure: Fig. 12.

Fig. 12. The designed wavefront aberration

Download Full Size | PDF

 figure: Fig. 13.

Fig. 13. The actual measured wavefront aberration

Download Full Size | PDF

In order to obtain higher accuracy of spatial information rectification, the area array camera and spectrometer need to carry out geometric rectification (the registration of photosensitive pixels when the same field of view is incident), which provides the datum data link for the final geometric rectification. The hyperspectral imager can complete the geometric calibration of spectral imaging detector and panchromatic array detector at the same time, which greatly simplifies the reference transmission link of geometric rectification. Through laboratory calibration, the position difference of two cameras’ focus and the angle deviation of projection center direction are detected. The spectrometer and area array detector are calibrated by rotating table, angular displacement table, six-dimensional adjustment table, collimator, plane mirror and theodolite. The photo of the hyperspectral imager with automatic geometric distortion rectification is shown in Fig. 14. Figure 15 shows the spectral image of a mercury lamp measured in the laboratory. Place the hyperspectral imager in front of the light exit of the integrating sphere for radiometric calibration, the imaging effect of the hyperspectral imager is shown in Fig. 16.

 figure: Fig. 14.

Fig. 14. The photo of the self-calibration hyperspectral imager

Download Full Size | PDF

 figure: Fig. 15.

Fig. 15. Mercury lamp spectrum image in laboratory

Download Full Size | PDF

 figure: Fig. 16.

Fig. 16. Results of laboratory radiometric calibration

Download Full Size | PDF

3.3 Test and analysis

The thin beam emitted by He-Ne laser is projected onto the imaging spectrometer after beam expansion and neutral filter attenuation. The measured spectrum is shown in Fig. 17, and its full width at half maximum is 2.3nm, that is, the actual spectral resolution of the system is 2.3nm, which meets the requirements of the design specification of 2.5nm.

 figure: Fig. 17.

Fig. 17. The measured spectrum of He-Ne laser

Download Full Size | PDF

The resolution target is used to test the geometrical resolution of the system. The beam transmitted from the resolution target is collimated by the collimator and received by the imaging spectrometer. The image of the target is obtained on the focal plane, as shown in Fig. 18(a). The figure shows the geometrical resolution detection results of the system under continuous spectrum illumination (the spatial frequencies from top to bottom are 8.00, 8.98, 10.10, 11.30, 12.70, 14.30 and 4.00 lp/mm). Figure 18(b) shows the DN value curve of the target board image along a central column. Therefore, the maximum spatial frequency Nc can be resolved is 14.3lp/mm. The focal length of the collimator $f_c^{\prime}$ is 41 mm, and the focal length of the system $f_p^{\prime}$ is 13 mm, so the spatial resolution of the system Np is,

$${N_p} = \frac{{{N_c}f_c^{\prime}}}{{f_p^{\prime}}} = 45.1\; lp\textrm{ / }mm.$$

Therefore, when the flying height is 100m, the ground geometric resolution is 8.5cm. The detector of hyperspectral imager adopts 16 bit AD quantization, and the detector noise level DN is about 3. The stray light level of hyperspectral imager has been reduced by setting the eliminating stray light aperture, coating the eliminating stray light black paint, and improving the reflectivity and transmittance of the optical element surfaces. Using a monochromator scan and measure stray light of other different wavelengths at specified wavelength position, the measured spectral stray light coefficient of the hyperspectral imager is 0.2% at 510nm.

 figure: Fig. 18.

Fig. 18. Image of resolving power test target (a) and intensity profile of test target (b)

Download Full Size | PDF

The device performance test was further carried out. Through indoor simulation experiments, the acquisition imaging quality and the fixed-speed flight stitching image quality were evaluated. The test site was the School of Information Science and Engineering, Ocean University of China. The test environment had no wind field interference and uniform and stable illumination. The device was mounted on an electric rotating platform. Simulate the spectral image collection situation of the UAV in the ideal flight state. The test scene size is about 100 meters (in the north-south direction) to 250 meters (in the east-west direction). As shown in Fig. 19, the hyperspectral imager is collecting data on the scene within the field of view. The data set consists of hyperspectral images and frame images. When the system collects data, the spectrometer and the area array detector use the same sampling frequency for imaging. Ensure that the spectral image and the area image are strictly registered in time and space. Figures 20(a) to 20(f) show six spectral images selected from the spectral cube, corresponding to wavelengths of 485, 595,610,675,730 and 800 nm. The hyperspectral image is stitched, and the effect is shown in Fig. 21. The image quality after splicing is ideal, which shows that the performance of the device is stable under constant speed flight mode of UAV.

 figure: Fig. 19.

Fig. 19. Self-calibration hyperspectral imager performance test

Download Full Size | PDF

 figure: Fig. 20.

Fig. 20. Spectral images of different bands

Download Full Size | PDF

 figure: Fig. 21.

Fig. 21. Spectral image after stitching

Download Full Size | PDF

Figure 22 is the collected data cube of hyperspectral image, and select the window, roof, wall and leaf area in the spectral cube. After multiple sampling and averaging, the data is processed to obtain the spectral curve, as shown in Fig. 22. Analyzing the spectral characteristics of leaves, we can see that there is a strong chlorophyll reflection peak near the wavelength of 550 nm, and the strong chlorophyll absorption band from 650 nm to 700 nm, which is basically consistent with the known experience.

 figure: Fig. 22.

Fig. 22. Hyperspectral data cube and reflectance spectrum curve of the region of interest

Download Full Size | PDF

4. Geometric rectification method and experiment

4.1 Geometric rectification method

In this study, an automatic geometric rectification workflow (Fig. 23) is presented for the integration of frame RGB images, push-broom hyperspectral scanner data and consumer-grade GNSS/INS navigation data for accurate geometric rectification of the hyperspectral scenes. As shown in the flowchart, basically, this registration process can be achieved automatically through three main steps: (1) Using low-quality navigation information from consumer-grade GNSS/INS units to perform modern automatic triangulation of overlapping frame images, which can provide the direct high-accuracy geo-referencing information for frame-based and hyperspectral push-broom scanner imagery; (2) The hyperspectral data together with the DEM and GNSS/INS-based geo-referencing information are processed to produce a partially-rectified hyperspectral-based orthophoto; (3) The geo-referencing parameters together with the DEM are used to produce a geometrically-accurate RGB-based orthophoto. resampling the partially-rectified hyperspectral orthophotos to the reference frame of the RGB-based orthophoto and co-registration with it. The geometric fidelity of the partially-rectified hyperspectral orthophoto will be improved. Suomalainen et al. compensated for the inferior quality of direct geo-referencing information through simultaneous integration of captured frame images and a Digital Elevation Model (DEM), which was derived from the frame images. Ramirez-Paredes presented a computer-vision approach for the indirect geo-referencing of the hyperspectral scenes. The geometric preprocessing of hyperspectral data is carried out by combining direct and indirect methods. The Digital Elevation Model is constructed by the method of triangulation interpolation, all calculations and conversions have been included in the relevant data processing and mapping software. Therefore, the process of using frame-based images to improve the geo-referencing information of the hyperspectral scanner scene mainly considers the method of identifying conjugate features in the image from different sensor patterns. Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Feature (SURF) and Harris corner detectors and descriptors are among the most commonly-used approaches. In this paper, the automated identification of conjugate points between the RGB-based orthophoto and the partially-rectified hyperspectral orthophotos is facilitated through a two-faceted matching strategy [15].

 figure: Fig. 23.

Fig. 23. Self-calibration technology flowchart

Download Full Size | PDF

Partially-rectified hyperspectral orthophotos collected as shown in Fig. 24, one can sequentially evaluate the parameters of the global affine transformation relating successive hyperspectral orthophotos. Tie points between the orthophotos from the first and second frame identified and used to estimate the parameters of the affine transformation T12 relating the reference frames of these partially-rectified orthophotos. The remaining affine transformation parameters between the resulting orthophotos can be estimated in a similar manner.

 figure: Fig. 24.

Fig. 24. The global affine transformation relating successive hyperspectral orthophotos

Download Full Size | PDF

4.2 Experiment

The existing hyperspectral data and POS data of UAV are geometrically rectified by using the above automatic geometric rectification process flow. The data set is obtained by the six-rotor UAV (DJI M600) through push broom, with a flight speed of about 2.5 m / s and an altitude of about 50 meters. Figure 25(a) is an unrectified hyperspectral image after splicing. It can be seen that the geometric deformation of the road and white target in the marked area in the image is serious, which makes interpretation difficult. The accuracy of the spectral data obtained by sampling needs to be verified. Figure 25(b) is the hyperspectral image obtained after geometric rectification using the above automatic rectification method. It can be seen from Fig. 25 that there are still some geometric deformations in the rectified image, such as the deformation of white target curve. Therefore, the contour of the white target in Fig. 26 is extracted for relative accuracy analysis. The deformation degree is shown in Table 3. Finally, the image geometric rectification accuracy is obtained: the line dislocation after rectification is less than 2 pixels.

 figure: Fig. 25.

Fig. 25. Comparison of original image and geometrically rectified image

Download Full Size | PDF

 figure: Fig. 26.

Fig. 26. The contour of white target is extracted for accuracy analysis

Download Full Size | PDF

Tables Icon

Table 3. Error analysis of target geometry rectification

The hyperspectral image collected by the drone is geo-registered and geometrically corrected with the coordinate and attitude information collected by GNSS/INS, and the values of Δx and Δy in the coordinate system are calculated to evaluate the geometric accuracy of the airborne hyperspectral image. As shown in Table 4, the geometric accuracy data of 6 hyperspectral images are used to compare the direct geo-referencing method based on GNSS/INS and the automatic geometric correction scheme based on frame images proposed in this paper.

Tables Icon

Table 4. Geometric errors of UAV hyperspectral images after geometric distortion rectification

5. Conclusion

Hyperspectral push-broom imaging method is very sensitive to the attitude and position changes of the UAV. In the imaging process, due to the influence of aircraft motion and air flow, the obtained hyperspectral data will have large geometric distortion, which will seriously affect the image quality, sometimes even cause the image to be unable to be visually interpreted, which seriously affects the normal application of the image.

An automatic geometric rectification workflow based on the hyperspectral imaging system is presented and verified in experiments, this method can use high-frame-rate, multi-overlap area array images to invert the attitude, position and orientation information of the drone platform and then perform geometric rectification on the hyperspectral images obtained synchronously. Specifically, the use of low-precision GNSS/INS unit and dense area array images can restore the camera posture during imaging. It makes up for the insufficient accuracy and sampling frequency of the GNSS/INS unit on the UAV. This paper proposes a specific method of automatic geometric rectification and introduces the technical process, design and develop a new compact hyperspectral imager. Completed the optical system optimization design, finite element model analysis, structural adjustment and modular integration. The integrated design of the imaging spectrometer and the area array detector with the common optical path can realize the synchronous acquisition of hyperspectral images and high frequency area array images. A FOV of 40 degree in the frame direction and 26 degree in the flight direction are realized with a focal length of 13mm, providing a large spectral range from 400 nm to 1000 nm, an excellent spectral resolution of 2.5 nm. The design result meets the requirements of image quality and the limited load limit of the UAV, and has the characteristics of small size and light weight. The hyperspectral imager was calibrated and geometrically verified, and functional verification and geometric rectification experiments were performed in the China Ocean University and the demonstration area.

This software and hardware integration research idea and technical solution is an effective method to solve the geometric rectification of the existing push-broom UAV hyperspectral remote sensing image, which greatly reduces the difficulty of users in the process of data acquisition and preprocessing. Laid the foundation for the next step to realize UAV hyperspectral remote sensing detection.

Funding

Jilin Scientific and Technological Development Program (20190302083GX); Key deployment project of the Marine Science Research Center of the Chinese Academy of Sciences (COMS2019J04); National Natural Science Foundation of China-Shandong Joint Fund (U2006209); National Defense Scientific Research Joint Cultivation Project (202065004); National Key Research and Development Program of China (2019YFC1408300, 2019YFC1408301); National Natural Science Foundation of China (52001295).

Acknowledgments

The authors would like to appreciate the technical editor and anonymous reviewers for their constructive comments and suggestions on this study.

Disclosures

The authors declare no conflicts of interest.

References

1. X. H. Wang and Q. S. Xue, “Optical system design of an atmospheric detector with nadir view and omnidirectional limb view,” Appl. Opt. 56(26), 7454–7461 (2017). [CrossRef]  

2. Q. S. Xue and M. Z. Duan, “Development of limb imaging spectrometer for atmospheric trace gas sounding,” J,” Guangxue Xuebao 33(5), 0522001 (2013). [CrossRef]  

3. A. Telmo, J. Hruka, and L. Pádua, “Hyperspectral Imaging: A Review on UAV-Based Sensors, Data Processing and Applications for Agriculture and Forestry,” Remote Sens. 9(11), 1110 (2017). [CrossRef]  

4. Y. K. Liu, C. R. Li, L. L. Ma, Y. G. Qian, N. Wang, C. X. Gao, and L. L. Tang, “Land surface reflectance retrieval from optical hyperspectral data collected with an unmanned aerial vehicle platform,” Opt. Express 27(5), 7174–7195 (2019). [CrossRef]  

5. B. P. Banerjee, S. Raval, and P. J. Cullen, “UAV-hyperspectral imaging of spectrally complex environments. International Journal of Remote Sensing,” Int. J. Remote Sens. 41(11), 4136–4159 (2020). [CrossRef]  

6. T. Adão, J. Hruska, L. Pádua, J. Bessa, E. Peres, R. Morais, and J. J. Sousa, “Hyperspectral Imaging: A Review on UAV-Based Sensors, Data Processing and Applications for Agriculture and Forestry,” Remote Sens. 9(11), 1110 (2017). [CrossRef]  

7. D. Deery, J. Jimenez-Berni, H. Jones, X. Sirault, and R. Furbank, “Proximal Remote Sensing Buggies and Potential Applications for Field-Based Phenotyping,” Agronomy 4(3), 349–379 (2014). [CrossRef]  

8. K. C. Lawrence, B. Park, W. R. Windham, and C. Mao, “CALIBRATION OF A PUSHBROOM HYPERSPECTRAL IMAGING SYSTEM FOR AGRICULTURAL INSPECTION,” Trans. Am. Soc. Agric. Eng. 46(2), 513–522 (2003). [CrossRef]  

9. Y. Inoue, E. Sakaiya, and Y. Zhu, “Diagnostic mapping of canopy nitrogen content in rice based on hyperspectral measurement,” Remote Sens. Environ. 126, 210–221 (2012). [CrossRef]  

10. J. J. Mitchell, N. F. Glenn, and T. T. Sankey, “Remote sensing of sagebrush canopy nitrogen,” Remote Sens. Environ. 124(124), 217–223 (2012). [CrossRef]  

11. J. Berni and P. J. Zarco-Tejada, “Thermal and Narrowband Multispectral Remote Sensing for Vegetation Monitoring From an Unmanned Aerial Vehicle,” IEEE Trans. Geosci. Remote Sens. 47(3), 722–738 (2009). [CrossRef]  

12. U. Beisl, “Absolute Spectroradiometric Calibration of the ADS49 Sensor,” Revue Francaise De Photogrammetrie Et De Teledetection 181, 14–18 (2012).

13. A. F. Habib, M. Morgan, and Y. Lee, “Bundle Adjustment with Self–Calibration Using Straight Lines,” Photogramm. Rec. 17(100), 635–650 (2002). [CrossRef]  

14. D. Turner, A. Lucieer, M. McCabe, S. Parkes, and I. Clarke, “Pushbroom Hyperspectral Imaging From An Unmanned Aircraft System (UAS) – Geometric Processingworkflow And Accuracy Assessment. ISPRS - International Archives of the Photogrammetry,” Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII-2/W6, 379–384 (2017). [CrossRef]  

15. H. Ayman, H. Youkyung, X. Weifeng, H. Fangning, Z. Zhou, and C. Melba, “Automated ortho-rectification of uav-based hyperspectral data over an agricultural field using frame rgb imagery,” Remote Sens. 8(10), 796 (2016). [CrossRef]  

16. J. Suomalainen, N. Anders, S. Iqbal, G. Roerink, J. Franke, P. Wenting, and L. Kooistra, “A Lightweight Hyperspectral Mapping System and Photogrammetric Processing Chain for Unmanned Aerial Vehicles,” Remote Sens. 6(11), 11013–11030 (2014). [CrossRef]  

17. Q. S. Xue, B. Yang, Z. Z. Tian, F. P. Wang, X. N. Luan, B. Mu, and S. R. Wang, “Spaceborne limb hyperspectral imager for ozone profile detection,” Opt. Express 27(22), 31348–31361 (2019). [CrossRef]  

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (26)

Fig. 1.
Fig. 1. The deviation of spatial information during data collection
Fig. 2.
Fig. 2. The geometric relationship between imaging spectrometer and area array camera
Fig. 3.
Fig. 3. Diffraction efficiency of triangular grooved grating
Fig. 4.
Fig. 4. Diffraction efficiency of rectangular groove grating
Fig. 5.
Fig. 5. Spot diagram on the image plane of telescope
Fig. 6.
Fig. 6. Optical path of hyperspectral image
Fig. 7.
Fig. 7. MTF of hyperspectral imager for different wavelengths: (a) 400 nm; (b)700 nm; (c) 1000nm
Fig. 8.
Fig. 8. The mechanical structure model
Fig. 9.
Fig. 9. Finite element model and boundary conditions
Fig. 10.
Fig. 10. The first three natural modes of hyperspectral system
Fig. 11.
Fig. 11. Setup and adjustment process of hyperspectral imaging system
Fig. 12.
Fig. 12. The designed wavefront aberration
Fig. 13.
Fig. 13. The actual measured wavefront aberration
Fig. 14.
Fig. 14. The photo of the self-calibration hyperspectral imager
Fig. 15.
Fig. 15. Mercury lamp spectrum image in laboratory
Fig. 16.
Fig. 16. Results of laboratory radiometric calibration
Fig. 17.
Fig. 17. The measured spectrum of He-Ne laser
Fig. 18.
Fig. 18. Image of resolving power test target (a) and intensity profile of test target (b)
Fig. 19.
Fig. 19. Self-calibration hyperspectral imager performance test
Fig. 20.
Fig. 20. Spectral images of different bands
Fig. 21.
Fig. 21. Spectral image after stitching
Fig. 22.
Fig. 22. Hyperspectral data cube and reflectance spectrum curve of the region of interest
Fig. 23.
Fig. 23. Self-calibration technology flowchart
Fig. 24.
Fig. 24. The global affine transformation relating successive hyperspectral orthophotos
Fig. 25.
Fig. 25. Comparison of original image and geometrically rectified image
Fig. 26.
Fig. 26. The contour of white target is extracted for accuracy analysis

Tables (4)

Tables Icon

Table 1. Technical specifications of imaging spectrometer

Tables Icon

Table 2. The first three natural frequencies and modes

Tables Icon

Table 3. Error analysis of target geometry rectification

Tables Icon

Table 4. Geometric errors of UAV hyperspectral images after geometric distortion rectification

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

N p = N c f c f p = 45.1 l p  /  m m .
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.