This study describes rapid prototype construction of small and lightweight push broom Hyper Spectral Imagers (HSI). The dispersive element housings are printed by a thermoplastic 3D printer combined with S-mount optical components and commercial off-the-shelf camera heads. Four models with a mass less than 200 g are presented with a spectral range in the visible to the near-infrared part of the electromagnetic spectrum. The bandpass is in the range from 1.4 - 5 nm. Three test experiments with motorized gimbals to stabilize attitude show that the instruments are capable of push broom spectral imaging from various platforms, including airborne drone to handheld operations.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
The push broom hyperspectral technique has been used for the last three decades [1,2] in industrial, airborne and satellite applications [3–6]. The push broom method records a perpendicular cross-section of any target along the track direction. In spectroscopy, the cross-section area corresponds to the entrance slit image as a function of wavelength, forming an image known as a spectrogram. The arrival of electronic image sensors such as the CCD (Charge Coupled Device) has made it possible to stack spectrograms at high speed to form a spectral data cube as we sweep the target. The number of reconstructed images from the data cube is only limited by the spectral bandpass and range of the instruments. The capability to form images as a function of wavelength is a powerful tool in remote sensing applications like on-line spectral identification, target recognition and classification.
However, the push broom technique requires line scanning of the target with high precision. For example, on an airborne or satellite platform, the instrument needs to be stabilized in attitude by a 3-axis gyro in order to sweep the target without spatial distortions along the direction of flight. Low cost Micro Electro Mechanical Systems (MEMS) Inertial Measurements Units (IMUs) with integrated 3-axis gyroscopes, accelerometers and magnetometers are found to be accurate for unmanned aerial vehicle navigation  and for motorized gimbal stabilization.
Other hyperspectral techniques have also emerged the last decade that applies Liquid Crystal Tunable Filters (LCTFs) [8, 9] and MEMS Fabry-Perot tunable interference filters  with direct imaging capability where no line scanning is necessary. The center wavelengths of the filters are controlled electronically. The only drawback of these techniques is low speed of the spectral sampling and shift of the target position during acquisition, both of which again requires attitude correction.
The recent developments in drone technology, CMOS (Complementary Metal Oxide Semiconductor) image sensors, gimbals based on MEMS IMUs and 3D printing open up new possibilities for construction of low cost push broom Hyper Spectral Imagers (HSI).
This work presents a do it yourself recipe for the construction of pocket size low mass hyperspectral push broom imagers. Only off-the-shelf optical elements, mechanical parts and electronics are used. The instruments are assembled by the use of a 3D printer. Three test applications are conducted to demonstrate that the instruments are capable of hyperspectral imaging.
2. 3D printing
When companies like MakerBot Industries started to mass produce 3D printers using thermoplastics as material back in 2009, the opportunity to create 3D objects became available to the public. Together with Computer Aided Design (CAD) software rapid prototyping of parts became possible. A number of studies have highlighted the use of 3D printing to produce optomechanical components  and spectrometers [12–14]. The latter studies did however not exploit the hyperspectral imaging capability.
The main constructional challenge in spectroscopy is the fact that the dispersive or refractive element, grating or prism, bends the input light beam. A machine with multiple axis of freedom is needed in manufacturing the part that holds the element. F. ex. a transmitting grating holder would require at least a 4-axis milling machine. A 3D printing machine uses Additive Manufacturing (AM) and can easily build parts with angular or more complex surfaces compared to conventional fabrication methods. In addition, the printing process produces parts by stacking honeycomb layers on top of each other, which gives high strengthand low mass. The latter is a clear advantage for low mass payloads aimed at drone operations.
Figure 1 panel (A) shows a 3D view of a simple transmission grating holder (Model 1). The solid modelling prior to 3D printing was carried out in Tinkercad, which is an online CAD program by the company Autodesk Inc. The design is based on a β = 19.36° wedge using a 600 lines/mm transmission grating with the collimated input beam entering at the bottom. The optical diagram is shown in Fig. 2. The design center output wavelength λ = 552.5 nm parallel to the wedge normal is then according to the grating equation  for the first spectral order (k = 1)
In order to avoid both gluing and threading, the holder can be split into two halves that encompass the optical element. Figure 1 panel (B) shows the Model 2, which is a snap-fit or push together design. The grating is half-way embedded into the holder. The same applies to the Collimator-slit-front-optics assembly and the detector lens. Due to the fact that thermoplastic materials tend to shrink during 3D printing, the trick is to scale up the parts 1-2% of their original designed size to make the embedded components frictional snap-fit fixed.
One obvious advantage is that parts may now be replaced without risk of damage and destruction. Nevertheless, the most brittle part is the grating, and to avoid damaging it, the slot height is designed 0.2 mm higher than the grating. A too close fit tends to break the grating when pushing the parts together. Thin slices of 0.2 mm thick rubber bands are used at the top and bottom of the slot to secure and fix the grating. Two straight through 45 mm long machine screws of diameter 3 mm holds the design together. All parts are frictional press locked and require no glue or epoxy when assembled.
3. Collimator-slit-front-optics assembly
The grating is collimated using the mix and match S-Mount components of the company Edmund Optics (EO) Ltd. The basic optical parts comprise one high precision slit, a collimator lens and a front lens. This requires three thin lens holders, one focus tube and one lock ring. An exploded view including the detector lens is shown in Fig. 3. Details of each part are listed in Table 1.
The thin lens holders of EO are designed for 10 mm diameter optical components. The useful optical aperture is 9 mm. The slit and the field lens are mounted together into one of the thin lens mounts. The slit is in contact with the flat side of the field lens. A second thin lens mount is used for the collimator lens that illuminates the grating. Both lens holders are mounted to a S-mount focus tube. The input f/value to the grating is f/3.3, which requires a f/2.5 camera lens in order to capture all the first spectral order (k = 1) light diffracted by the grating. The grating is larger than required in order to only use the center part that is not affected by frictional damage when sliding it into its holder. The separation between the center of the grating and the center of two lenses is only 5 mm. The third lens holder is used in front of the slit to provide sufficient threads and focus distance for the front lens and its lock nut. The f/4 front lens is the main aperture stop of the system, which prevent field of view overfill of the collimator lens.
Figure 4 shows the assembly of the Collimator-slit-front-optics, grating and detector lens embedded into one half of the holder. We used the MakerBot’s Replicator 2 armed with PLA (Polylactic Acid) thermoplastics filament to print the part.
The slit height magnification at the sensor focus plane is 1.28 due to the field lens in the slit entrance plane and the spectral bandpass is FWHM = 1.4 nm (Full Width at Half Maximum). The visible part of the electromagnetic spectrum (400-700 nm) illuminates an area in the focus plane of approximately 4.8 × 3.2 mm2, which means that a 1/3” sized image sensor can be used.
Several prototypes have been assembled with different low cost camera heads. Four of them are shown in Fig. 5. A miniature color CCD (Charge Coupled Device) video camera head (Turnigy 600 TVL) used by the Radio Control (RC) community, a CamOne Infinity action camera, and two monochrome industrial CMOS camera heads are tested out as potential detectors.
The main differences between the prototypes are the type of detectors and how they are mounted to the grating house. Key instrumental parameters are summarized in Table 2. The optical parts are identical, except for instrument (D) where the slit width is increased to 75 μm and the collimator lens is decreased in focal length to 25 mm. The latter is done to increase throughput and sensitivity.
The camera head of instrument (A) is snap fitted by side plates that are added to the 3D print model. A wireless video transmitter holder is also included. For instruments (B) and (C), 3D printed side plates are mounted to press fit the camera heads. The two 45 mm long bolts that go straight through the grating holder (Model 2) are used for this purpose. A metal strip is added to instrument (C) to improve the design. The same technique is used on instrument (D), except that the side plates are aluminum. The camera head is now mounted to the plates by 4 x 3 mm diameter machine screws. The aluminum plates also act as heat sink to the camera head.
5. Alignment and calibration
The instrument is focused by first mounting the camera lens to the detector. Brass spacer rings are used to obtain a fixed back focal length of 17.5 mm, required for C-mount camera heads to be focused at infinity. Secondly, after mounting all parts together, spectral focus is achieved by tuning the S-mount focus tube. A spectral line source such as a fluorescence tube illuminating a diffuse surface is used to identify spectral lines as sharp as possible. Thirdly, the front lens is focused by identifying sharp structures in the horizontal direction of the spectrograms, which originate from any sharped edged target object located at infinity as seen by the font lens.
A spectrogram of a white paper edge illuminated by an OSRAM fluorescent tube is shown in Fig. 6. The horizontal edge across the spectrum should be as sharp as possible to obtain focus of the front lens. Note that the edge is sharper at the wavelength center of the spectrogram. This is believed to be due to chromatic aberration from the collimator lens. An achromatic lens should be tested in future designs. The spectral lines are also slightly curved and tilted ( ± 0.45 nm) due to astigmatism and a small misalignment of the slit angle, respectively. The wavelength calibration is therefore conducted by a polynomial fit of the known emission line peaks in the center row spectrum of a Mercury vapour tube spectrogram. As noted above, any misalignments of the slit and astigmatism effects will produce errors in the order of ± 0.5 nm.
It should be emphasized that the use of color sensors is not necessary in spectroscopy, but the low cost and availability make them ideal for testing on prototypes. In addition, the colored spectrograms are useful for pedagogic and demonstrational reasons. For further processing, the spectrograms are simply converted to grayscale by adding the response from each color channel and scaling the net result down to 8-bit. This is not ideal for target intensity work since we do not de-mosaic the raw response from the Color Filter Array (CFA) in front of the sensor surface. Instrument (A) and (B) are as a consequence not intensity calibrated.
Instruments (C) and (D) use monochrome CMOS sensors that are also sensitive in the Near-Infrared (NIR) region of the spectrum. However, based on our calibration results, the effective or useful spectral range is only from 400 to 800 nm. These instruments are not sensitive to Ultra Violet (UV) light below 400 nm due to absorption by the glass lenses, and the second spectral order (k = 2) appears or blends in above 800 nm. The effect is visualized in Fig. 7 in the wavelength calibration of instrument (D).
The first order emission lines of Mercury (Hg) at wavelength 404.7 and 435.8 nm reappear above 800 nm as second order lines. This opens the opportunity to increase readout time by cropping the useful part of each spectrogram, and cutting the second order part in NIR and the non-illuminated areas in UV out. The latter could also be used to detect the background level of the sensor.
Sensitivity calibration was conducted by the use of a standard certified irradiance Tungsten lamp and a Lambertian screen as target . The raw 8-bit count response and source spectrum in units of mW m−2 nm−1 of the screen are shown in Fig. 7. Certification above 800 nm requires an order blocking filter with cutoff wavelengths above 400 nm. A motorized filter wheel could be installed in front of the grating to handle this issue, but it would complicate the design, introduce moving parts and add mass. Another solution would be to sacrifice spectral resolution and use a prism instead of the grating with no overlapping spectral orders.
6. Multi-purpose examples
The following three test applications are setup to demonstrate that the constructed HSI are capable of forming multi spectral images where the target objects are visually identified. Lessons learned are also included.
6.1 Qctocopter drone experiment
The first airborne test with HSI - instrument (A) was conducted as part of the field work for the Arctic Earth Observational summer school hosted by the Norwegian Centre for Space-related Education (NAROM) at the Andøya Space Centre (ASC) in mid-August 2016. The experiment used an Octocopter operated by the Remotely Piloted Aircraft (RPA) group. The target area was a field of vegetation close to an old military camp in Skarsteindalen, which is approximately 12 km south of Andenes city in Norway.
The Octocopter was equipped with a motorized 2-axis gyro platform for the payload. The gyro stabilized pitch and roll attitude angles during the flight. Note that the yaw or heading of the payload was not stabilized. In addition to the HSI – instrument (A), a Normalized Difference Vegetation Index (NDVI) camera was installed to map the terrain and the vegetation . The camera is a modified Canon PowerShot SX260. The original CFA is replaced by a Near Infrared (NIR) sensitive CFA by the workshop MaxMax.com - Llewellyn Data Processing (LDP LLC). The NDVI camera stored 1/1000 second exposures to an internal memory card every 2 seconds. Each image was tagged with GPS position data. The video from the HSI was transmitted and received by a video recorder at ground level. The exposure time was set to 1/50 second. The live stream was 25 frames per second. The automatic gain of the camera was enabled.
The main objective of the experiment was to determine whether the drone and the gyro together could act as a line scanning platform for the HSI instruments. A stable attitude flight where the image in the slit entrance plane is moved parallel along the slit height axis and perpendicular to the flight direction, should in theory sample the target in a linear manner without spatial disturbance.
A geo-corrected orthomosaic image constructed from the NDVI image sequence is used to compare and detect similar spatial features obtained from the hyperspectral data cube. The mosaic is visualized in Fig. 8. Five white diffuse colored surfaces with a square area of 1 m2 are used as ground control points for calibration of the image position. Each control point position is measured by a Differential Global Positioning System (DGPS) prior to the airborne campaign. Geolocation and rectification were carried out by the students as part of their training.
The pilot operated the drone close to the white van seen at the parking lot to the left in the middle of Fig. 8. The flight pattern is shaped like a fan. Yellow boxes mark the sampling area of the HSI. For each track the drone was flown out into the field in different directions in order to return with low velocity back towards the position of the pilot. The pilot manually controlled the aircraft in altitude hold mode. More advanced intelligent modes like automatic GPS waypoint navigation was at this point in time not an option. The altitude ranged from 100 to 300 m with ground track speed velocities from 1.7 to 3.1 m/s. Note that the weather conditions were not optimal due to wind gusts up to 5 m/s, especially at the high altitude range. As a consequence, data from only 4 out of 25 tracks were selected for further analysis based on stable flight performance of the drone.
The comparison between the NDVI and the HSI camera is shown in Fig. 9. The NDVI images are first scaled down to 30% of their original size. The HSI images are color RGB composites. Each color channel is constructed by stacking the accumulated response in wavelength across a vertical sliced box of the video frames (spectrograms). The position and width of the box defines the center wavelength and the image bandpass, respectively. The blue, green and red channels of the composite HSI images are centered at 470, 550 and 630 nm, all with a bandpass of 10 nm. Secondly, the color images are scaled down to match the spatial scale of the NDVI images. The adjustment of the scale was done by visual examination of the resulting match.
It is clear that the HSI is capable of reproducing the same features and objects that the NDVI camera detects. The main errors are occasional horizontal stripes associated with wind gusts, and scaling in the flight direction. The latter is due to variations in aircraft velocity and flight direction (yaw).
6.2 Handheld gyro stabilized experiment
The next experiment was conducted with HSI-instrument (B). The imager was mounted on a 3-axis gimbal made by the company Feiyu Tech, model MG. The gimbal is designed for handheld operation of Digital Single Lens Reflex (DSLR) cameras. Integrated 3-axis accelerometers and gyroscopes feed the three brushless motors with attitude data (tilt, roll and pan). The HSI was mounted to an L-shaped aluminum mount bar that increased the mass with 90g. It was necessary to add mass to the gimbal in order to balance all 3-axis correctly. The HSI recorded the spectral movie on two internal 32G Bytes memory cards installed in theCamOne Infinity action camera head and the gimbal was powered by its own batteries, making the assembled system self-contained with no auxiliary connections or computers.
A panoramic sweep with the gimbal is shown in Fig. 10. The target is the houses of Longyearbyen on the 25th of September 2017. The HSI and the gimbal were stationary mounted on the roof of The University Centre in Svalbard (UNIS) with the yaw-axis panning from East to West up the Longyearbyen valley. Nine images were extracted from the hyperspectral data cube every 30 nm from 470 to 670 nm center wavelength. All images have a bandpass of 10 nm. The total sweep took 152 seconds. The RGB composite was constructed using center wavelengths at 480nm (blue), 550 nm (green) and 620 nm (red).
As expected, the HSI produced a color image close to visual perception for all targets in the scenario. The images seem to be more blurry in the blue part compared to the green-red part of the spectrum. The effect is mainly due to chromatic aberration as explained in Section 5. Note that the images are scaled a factor of 3 down in the vertical direction and equalized in intensity to increase contrast and brightness by the program paint.net from dotPDN LLC.
6.3 Computer screen test experiment
In the last experiment, the gimbal and the HSI – instrument (B) was used handheld to sweep a computer screen displaying an image of a fruit collection. 571 spectrograms were recorded in 22 seconds at a distance of 1.5 meters from the computer screen. The target screen color photo and the HSI RGB composite are shown in Fig. 11. Center wavelengths at 490 nm (blue), 552 nm (green) and 620 nm (red) were used to construct the image. Image bandpass is 10 nm for all channels. The image is resized to match the target size. The vertical and horizontal scale factors are 0.22 and 1.38, respectively. The horizontal black line close to the top in the image is due to dust on the slit.
There are two advantages of using a computer screen as test target. First, there is no need for an additional reference camera to document the target. Secondly, the target is illuminated by the screen itself.
The experiment shows that the Gimbal and the HSI together are capable of reproducing both color and shape of the target objects in the scene. The focus of the HSI RGB composite is not as sharp as the test image. The size of the slit width requires a down scaling in the vertical direction in order to obtain equal sized or square image pixels. In addition, multi-colored and non-smooth transitions are seen at the edges of the objects. The effects are mainly due to chromatic aberration and operator shake, respectively.
The sensitivity of the HSI prototypes has not been a limiting factor in the study. As mentioned, higher throughput may be achieved by increasing the slit width and decreasing the collimator focal length to 25 mm. A faster front lens at f/2.8 may then be used. As a result both the spectral and the spatial resolution would be degraded. Future studies will be conducted to optimize performance bases on the level of target illumination and spectral calibration.
7. Concluding remarks
Together with off-the-shelf optical components, low cost camera heads and 3D printing, it is possible to construct a push broom hyperspectral camera with a mass less than 200 g and cost below 700 US$. Laboratory calibrations supplemented with field experiments have proven the conceptual design based on visual comparison in color and shape with composite camera images recorder simultaneously, which is believed to be sufficient proof at this stage of the development.
The principal results obtained in this study can be summarized as follows.
- (1) 3D printing is a powerful tool for construction of rapid prototype parts with sufficient accuracy for optical systems, like a transmission grating house presented in this study. The lightweight and strength of the printed thermoplastic material is an advantage when optimizing payload mass and size, especially for drone operations.
- (2) The accessibility of mass produced low cost camera sensors developed for the industry, the Radio Control (RC) model community and action camera users, makes it possible to record visible and even the near infrared part of the electromagnetic spectrum at high frame rates.
- (3) Line scanning or push broom hyperspectral imaging from an airborne platform or simply handheld can be done with a commercial off-the-shelf motorized gyro stabilized gimbal.
- (4) Self-contained motorized gyro gimbals and internal camera head recording reduce auxiliary device support requirements and complexity of field operations.
- (5) The push broom hyperspectral imaging technique is revitalized.
The Research Council of Norway (RCN) (270959, 223254).
We would like to thank the students Joy Muraszko, Henna-Reetta Hannula, Stefan Ram, Alistair McConnel, Elena Berndt, Daniel Kepski, Sara Marie Strand, Birgitte Madsen, Daniel Nilsson, Eivind Kolås, Leif Erik Andersson, Josh Marks, Nicole Nagy, Anne Kathrine Wenaas Ribe, Caelia Gardiner, Christopher Cosgrove, Conor Bolas, Maximilian Fuermann and Yitayew Temesgen Gebrie of the Arctic Earth Observational summer course which was successfully hosted by Christoffer Stausland from the Centre for Space-related Education (NAROM) at the Andøya Space Centre (ASC) in 2016. Also thanks to Mats Mikalsen Kristensen and Jostein Sveen from the Remotely Piloted Aircraft (RPA) group at ASC.
References and links
1. G. Vane, Imaging Spectroscopy II (SPIE, 1987).
2. W. L. Wolfe, Introduction to Imaging Spectrometers (SPIE, 1997).
3. E. Herrala and J. Okkonen, “Imaging spectrograph and camera solutions for industrial applications,” Int. J. Pattern Recognit. Artif. Intell. 10(01), 43–54 (1996). [CrossRef]
4. T. S. Hyvarinen, E. Herrala, and A. Dall’Ava, “Direct sight imaging spectrograph: a unique add-in component brings spectral imaging to industrial applications,” Proc. SPIE 3302, 165–175 (1998). [CrossRef]
5. F. Sigernes, K. Heia, H. Nilsen, and T. Svenøe, “Imaging spectroscopy applied in the fish industry?” Norw. Soc. Image Process. Pattern Recogn. 2, 16–24 (1998).
7. J. Hosen, H. H. Helgesen, L. Fusini, T. A. Johansen, and T. I. Fossen, “A Vision-aided Nonlinear Observer for Fixed-wing Unmanned Aerial Vehicle Navigation,” J. Guid. Control Dyn. 39(8), 1777–1789 (2016). [CrossRef]
8. P. J. Miller, “Use of tunable liquid crystal filters to link radiometric and photometric standards,” Metrologia 28(3), 145–149 (1991). [CrossRef]
9. F. Sigernes, Y. Ivanov, S. Chernouss, T. Trondsen, A. Roldugin, Y. Fedorenko, B. Kozelov, A. Kirillov, I. Kornilov, V. Safargaleev, S. Holmen, M. Dyrland, D. Lorentzen, and L. Baddeley, “Hyperspectral all-sky imaging of auroras,” Opt. Express 20(25), 27650–27660 (2012). [CrossRef] [PubMed]
10. J. Antila, U. Kantojärvi, R. Mannila, A. Rissanen, I. Näkki, J. Ollila, and H. Saari, “Spectral imaging device based on a tuneable MEMS Fabry-Perot interferometer,” Proc. SPIE 8374, 8374 (2012). [CrossRef]
12. M. Arafat Hossain, J. Canning, S. Ast, K. Cook, P. J. Rutledge, and A. Jamalipour, “Combined “dual” absorption and fluorescence smartphone spectrometers,” Opt. Lett. 40(8), 1737–1740 (2015). [CrossRef] [PubMed]
13. T. C. Wilkes, A. J. S. McGonigle, J. R. Willmott, T. D. Pering, and J. M. Cook, “Low-cost 3D printed 1 nm resolution smartphone sensor-based spectrometer: instrument design and application in ultraviolet spectroscopy,” Opt. Lett. 42(21), 4323–4326 (2017). [CrossRef] [PubMed]
14. L.-J. Wang, Y.-C. Chang, R. Sun, and L. Li, “A multichannel smartphone optical biosensor for high-throughput point-of-care diagnostics,” Biosens. Bioelectron. 87, 686–692 (2017). [CrossRef] [PubMed]
15. C. Palmer and E. Loewen, Diffraction Grating Handbook (Thermo RGL, 2000).
16. F. Sigernes, J. M. Holmes, M. Dyrland, D. A. Lorentzen, S. A. Chernous, T. Svinyu, J. Moen, and C. S. Deehr, “Absolute calibration of optical devices with a small field of view,” J. Opt. Technol. 74(10), 669–674 (2007). [CrossRef]
17. W. Nijland, R. de Jong, S. M. de Jong, M. A. Wulder, C. W. Bater, and N. C. Coops, “Monitoring plant condition and phenology using infrared sensitive consumer grade digital cameras,” Agric. For. Meteorol. 184, 98–106 (2014). [CrossRef]