Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

High-robustness autofocusing method in the microscope with laser-based arrayed spots

Open Access Open Access

Abstract

Accurate and rapid autofocus technology plays a crucial role in various fields, including automatic optical inspection technology, bio-chips scanning, and semiconductor manufacturing. The current photoelectric autofocus methods have limitations because of detecting the focal plane solely at the center of the microscope field of view. In the application of Stereo-seq the risk of autofocus errors will be increased, which have reduced the robustness of the system, like when the surface of the tested samples are wrinkling and inconsistent thickness, or the detection spot is at the edge of the sample. To enhance the robustness of the autofocus system and mitigate the constraints of the photoelectric autofocus methods, the laser-based arrayed spots photoelectric autofocus method has been proposed. To achieve the uniform light splitting, a 2D-Dammann grating is incorporated into the optical path of the autofocus system, resulting in the formation of an n × n arrayed spots on the surface of the sample. Through experimental verification, it has been demonstrated that this method can achieve the autofocus range of ±100μm and the autofocus accuracy of ±1/4 DOF when applied to a microscope equipped with a 10× objective lens, thereby satisfying the requirements for microscopic focusing. The arrayed light autofocus method devised in this study presents what we believe is a novel research concept for active autofocus detection and holds significant application value.

© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

In recent years, autofocus technology has played an increasingly significant role in various application fields [1], including automatic optical inspection, TFT arrays scanning, semiconductor manufacturing, and biological imaging [2,3]. In the fields such as bio-chips scanning and semiconductor manufacturing, the size of the sample area is generally larger than the microscope's field of view. Therefore, scanning imaging technology is commonly employed, which involves completing the exposure imaging of a sample area and then moving the sample step by step through the stage to bring the adjacent area into the imaging field of view for another exposure. However, during this process, defocusing may occur due to mechanical errors, platform movement, or sample flatness errors. Relying solely on manual focus not only slows down the scanning speed but also introduces human errors. To address these challenges, an autofocus system is utilized. This system performs real-time scanning to detect defocus errors of the sample during the step scanning movement of the motion-stage. It then controls the movement of the motion-stage to compensate for the defocus amount in real time. This ensures that the sample remains in the optimal focus of the microscope objective throughout the imaging process.

Autofocus solutions can be broadly categorized into two methods: image-based autofocus detection and photoelectric-based autofocus detection. The image-based autofocus detective method determines the focus position by analyzing the sharpness or the spatial frequency function of the overall image within the microscope field of view to obtain focus information [410]. This method is capable of detecting surface undulations of complex samples, enabling automatic focusing on different positions within the same field of view. However, due to the requirement of image acquisition and processing, this method has low autofocused efficiency which is not suitable for high-speed scanning imaging technology.

Another autofocus method is based on photoelectric detection technology [1,1116]. This method typically employs laser or LED light sources to certain the defocus distance and direction of the sample in order to rectify the focal plane variation so that we can achieve rapid and precise focal plane positioning. Nevertheless, autofocus methods based on photoelectric detection are often limited to single-point autofocus, with the focal point situated at the center of the field of view on the sample surface. Therefore, this method can only determine the focal plane of the central spot position within the field of view.

Stereo-seq utilizes microscopic scanning imaging to identify the anatomical structure of tissue sections, facilitating the production of high-definition, comprehensive transcriptome atlases. It is capable of capturing tissue section areas up to a few square centimeters [17]. The photoelectric autofocus system is critical in the swift imaging scans of Stereo-seq. Nonetheless, the preparation of large tissue sections frequently results in issues like wrinkling or inconsistent thickness, which leads to errors in focus judgment with conventional single-point autofocus systems. Moreover, the mapping of Stereo-seq necessitates the scanning of edges of samples, yet single-point autofocus systems tend to detect the focal planes of areas lacking biological tissue, leading to the failures of the autofocus feature.

In the past, various solutions have been proposed to address the issue. One approach suggested was the incorporation of an adjustable lens (typically the liquid lens) to alter the position of the focal plane and focus on different planes. Alternatively, we could adjust the optical axis of the relay lens in the autofocus optical system [1820]. However, these solutions would either increase the complexity of the autofocus system or necessitate modifications to the microscope itself. Another solution involved programming a scan pre-modeling process to handle focus anomalies [21]. Nevertheless, this method required manual intervention and was comparatively less efficient.

Consequently, we have developed an innovative technique for arrayed spots autofocus. This research enables the detection of defocus at any position within the microscopic field by generating the n × n autofocus detection light array on the surface of the sample. Utilizing this method, we have effectively resolved the issues of focus failure due to sample fabrication defects or scanning positions, substantially improving the robustness of conventional photoelectric autofocus techniques. This method ensures both the efficiency and extensive range of photoelectric autofocus without adding to the complexity of the detection system, offering a viable solution for industrial applications.

Moreover, aberrations are an unavoidable aspect between the autofocus system and the microscopy system in practical scenarios. These aberrations affect the shape of the autofocus spots near the focal surface, which in turn impacts the accuracy of the autofocus system and the linearity of the autofocus curve. To address this issue, we added an appropriate amount of defocus to the autofocus system. Under these circumstances, the defocus has a significant effect on the autofocus spots, which overshadows the degradation of the spots caused by system aberrations. Consequently, the CCD captures a relatively standard semi-circular spot pattern. Experimental results confirm that this method effectively improves the linearity of the autofocus curve and the precision of autofocus system.

2. Arrayed spots autofocus system for microscope

Laser-based autofocus technology has been extensively employed in the autofocus of microscopy systems [2225]. This technique leverages a CCD sensor to capture an image of a sample's reflected laser beam and generates a signal based on the center-of-mass position within the image.

The schematic diagram of the optical path principle of the proposed automatic focus detection system is depicted in Fig. 1(a) [26]. According to the principles of geometric optics, when the laser irradiates point A on the sample surface, which is located on the focal plane of the objective lens, it is reflected and then falls on point A’ of the focusing image sensor. At this particular point, we are able to obtain the following information:

$${\mathrm{tan\alpha}} = \textrm{d}/{\textrm{f}_1}$$

The expression for AA1 can be derived based on the given information. Let α denote the semi-aperture angle of the microscopic objective lens, which represents the maximum angle after the beam parallel to the optical axis is refracted by the objective lens. Let d represent the spot radius of the collimated laser beam, and let f1 represent the focal length of the microscopic objective lens.

 figure: Fig. 1.

Fig. 1. (a) Schematic diagram of the optical path for the proposed autofocus microscope system. The green and blue beams represent the beams reflected up and below the focal plane, respectively. The red beam is the beam reflected on the focal plane. δ represents the defocus amount, Δ denotes the displacement of the focusing image at the CCD detector, f1 and f2 represent the focal length of the objective lens and imaging lens, respectively. d is the spot radius of the collimated beam, and α is the semi-aperture angle of the microscope objective. (b) Image of the reflected laser beam captured from the CCD sensor in theory. Δ represents the spot radius. The blue dot and red dot signify the center of the circle and the centroid, respectively. Their corresponding coordinate are (Xc, Yc) and (Xcentroid, Ycentroid), respectively.

Download Full Size | PDF

When the sample is positioned at a distance of ±δ from the focal plane, the laser beam is directed at point C on the surface of the sample. The reflected beam intersects with A1 at the focal plane and then falls onto point A1’ of the focus-checking image sensor. At this particular configuration, AA1 can be expressed as:

$$\textrm{A}{\textrm{A}_1} = 2{\mathrm{\delta tan\alpha}}$$

The displacement of the point at the focus-checking image sensor is:

$$\Delta = \textrm{kA}{\textrm{A}_1}{\; }$$
where k represents the total magnification of the achromatic lens and the microscope objective. This value is determined by the ratio of the focal length f2 of the achromatic lens to the focal length f1 of the microscope objective.
$$\textrm{k} = {\textrm{f}_2}/{\textrm{f}_1}$$

The simultaneous Eqs. (1)–(4) ultimately yield a linear relationship between defocus amount and displacement of light on the image sensor:

$$\Delta = \mathrm{\delta }\frac{{2\textrm{d}{\textrm{f}_2}}}{{{\textrm{f}_1}^2}}$$

The focus-detection measurement value refers to the displacement of light on the focus-detecting image sensor. In order to enhance autofocus efficiency, it is necessary for the focus-detection measurement value to have a linear relationship with the defocus amount. Geometric optics suggests several commonly used measurement values for focusing, including centroid of spot, distance between geometrical center and centroid of spot, and spot radius.

Figure 1(b) illustrates the morphology of the laser spot on the CCD sensor under varying conditions, as well as the efficacy of the focus-detection measurement value on the spot. In the Fig. 1(b), Δ represents the laser spot radius. The positions of the geometric center and the centroid of the image captured from the CCD sensor are denoted as (Xc, Yc) and (Xcentroid, Ycentroid) respectively. The geometric image center (Xc, Yc) remains unchanged, whereas the centroid of the image (Xcentroid, Ycentroid) can be mathematically expressed as:

$${\textrm{X}_{\textrm{centroid}}} = \frac{{\sum \sum ({\textrm{X} - {\textrm{X}_\textrm{C}}} ){\textrm{P}_{\textrm{ij}}}}}{{\sum \sum {\textrm{P}_{\textrm{ij}}}}}$$
$${\textrm{Y}_{\textrm{centroid}}} = \frac{{\sum \sum ({\textrm{Y} - {\textrm{Y}_\textrm{C}}} ){\textrm{P}_{\textrm{ij}}}}}{{\sum \sum {\textrm{P}_{\textrm{ij}}}}}$$

In Eqs. (6) and (7), the variables i and j represent the row number and column number of the CCD sensor respectively. The variable Pij represents the gray value of the image corresponding to the pixel with coordinates (x, y). Assuming uniform gray value throughout the entire image, there is a linear correlation between the centroid of the image and the defocus amount δ on the CCD sensor. Therefore, it can be inferred that the centroid or radius of the reflected laser beam's image, as captured by the image sensor, exhibits a linear variation with the defocus amount of the microscope. This variation can be used as a suitable feedback signal for the closed-loop position control system.

The paper intends to utilize the spot radius as the measured value, as the superposition calculation of the center of mass in Eqs. (6) and (7) requires some time. On the other hand, the spot radius value only necessitates recording the intensity of luminous distribution of the CCD detector. Hence, this method not only guarantees autofocus accuracy but also achieves high autofocus speed.

The basic structure of the arrayed spots autofocus system of microscope proposed in this paper is illustrated in Fig. 2(a). The system utilizes a semiconductor laser with a wavelength of 660 nm as its light source. The visible light is employed for the autofocus light source to facilitate the maintenance and observation of the autofocus module. To address the issues concerning the beam quality and light intensity uniformity of semiconductor lasers, we employs a spatial filter positioned after the laser. This spatial filter helps generate spots with uniform light intensity. The incident beam undergoes a transformation process through the beam expander and diaphragm, resulting in the formation of a semicircular beam. Subsequently, the beam is divided by spatial light modulator (the paper employs a two-dimensional Dammann grating to divide a beam of light into the 3 × 3 array). Two beam splitters are then utilized to guide the laser light emitted from the second splitter through the objective lens and onto the sample surface, thereby generating an array of illumination spots. Upon reflection from the sample, these beams are once again directed through the objective lens and two beam splitters before reaching the imaging lens. Ultimately, they are captured by the CCD sensor. It is worth noting that the shape of the laser spots on the CCD detector is also semi-circular.

 figure: Fig. 2.

Fig. 2. (a) Detailed structure of proposed auto-focusing microscope features a comprehensive design that includes several key components. One of the primary modifications is the addition of a spatial filter at the initial autofocus system light source to even out the light distribution. Furthermore, a spatial light modulator is incorporated after the semicircular diaphragm to function as a light splitting device (the paper employs a two-dimensional Dammann grating to divide a beam of light into a 3 × 3 array). In addition to the structural addition, the CCD detector is also relocated towards the imaging lens which provides defocus for the autofocus detection spots. The microscope has three light paths: the blue light path for imaging, the red light-path for incident light used for focusing, and the green light path for reflected light used for focusing. The autofocus module is represented by the dotted line section. Beam splitter 1 is a semi-transparent mirror, while beam splitter 2 is a dichroic mirror. The lens is used for imaging, and the knife can also function as a semicircular diaphragm. (b) ZEMAX optical model of proposed autofocusing microscope. Red, blue, and green are the respective colors of the three beams of light that are separated by the two-dimensional grating. These beams are then reflected by the sample before reaching the CCD detector.

Download Full Size | PDF

After the signal processing module processes the light spots, the arrayed light spots acquire the defocus amount at different positions of the sample. Subsequently, the defocus signal is transmitted to the worktable motor, which drives the sample to move up and down along the optical axis direction in order to achieve focusing.

When the sample being measured is in close proximity to the focal plane, the light spot detected by the CCD sensor is influenced by aberration, resulting in significant distortion. Moreover, the deformation of the autofocus light spot near the focal plane is minimal. Both of these factors lead to errors in determining the defocus distance and direction. The research proposes moving the CCD detector towards the imaging lens, causing the autofocus light reflected by the objective lens's focal surface to deviate from the focus surface of the imaging lens. This method eliminates the impact of aberration on the focus detection system when the sample is near the microscope objective focal plane with improving the system's focus accuracy and reducing auto-focusing time.

Figure 2(b) shows the optical model of the autofocus system constructed using optical design software. The simulation light source employs a 660 nm Gaussian beam with a beam diameter of 3 mm. The microscope objective is a 20× objective lens with an approximate focal length of 10 mm. The imaging lens has a focal length of 100 mm, while the CCD sensor is positioned 80 mm behind the imaging lens. Figure 3 displays the simulation of footprint diagram imaged on the CCD sensor and sample surface, respectively under varying defocus amounts.

 figure: Fig. 3.

Fig. 3. The fig. (a), (b), and (c) depict the footprint diagram detected by the CCD sensor at defocus amounts of -50µm, 0µm, and 50µm, respectively. Meanwhile, the figures (d), (e), and (f) illustrate the footprint diagram present on the sample surface at -50µm, 0µm, and 50µm defocus. (The imaging lens has a focal length of 100 mm, and the CCD sensor is positioned 80 mm behind the imaging lens, which serves as a preset defocus amount for the focusing system.)

Download Full Size | PDF

By constructing a laboratory prototype, depicted in Fig. 4, the effect of the simulated autofocus microscope system has been verified in this study. The prototype employs a 660 nm low-noise semiconductor laser as its light source, with a spatial filter consisting of a small 2mm-diameter aperture. Furthermore, a 3× beam expander from Thorlabs is utilized, along with a mirror serving as the sample. The spatial light modulator utilized in this study is a two-dimensional Dammann grating produced by HOLO/OR, a leading manufacturer in the field. The grating can generate a 3 × 3 array of beams, with the splitting angle of 0.79°×0.79°. The grating exhibits an impressive total efficiency of 72% and maintains a contrast ratio below 15%. To assess the suitability of the designed autofocus system, microscope objectives with magnifications of 10×and 20× are integrated into the infinity corrected optical system. To ensure a more pronounced variation in the autofocus spots on the CCD detector, the focal length of the imaging lens is extended to 180 mm, while maintaining a distance of approximately 160 mm between the imaging lens and the CCD sensor. The CCD features a unit pixel size of 3.45µm and a sensor size of 14.2 × 10.4 mm, guaranteeing a sufficiently high level of focus accuracy and range.

 figure: Fig. 4.

Fig. 4. A photograph of a laboratory-built prototype is shown. The autofocus module is indicated by the green frame line, while the remaining components consist of the infinity correction microscope parts. The red light is the incident focusing light, and the blue light is the reflected focusing light.

Download Full Size | PDF

The defocus of the prototype is adjusted by the movement of the objective lens. The objective lens shift-stage used is P725.4CA from PI Company, and image processing is carried out using XIMEA CAMTOOl. Spot analysis and extraction of focus measurement values are achieved through Matlab algorithm.

Figures 5(a) and 5(c) illustrate the notable enhancement in quality of autofocus spot image following the addition of a spatial filter subsequent to the laser source. At the same time, Figs. 5(b) and 5(d) contrast the autofocus spot from a standard single-point autofocus system with the arrayed spots resulting from the incorporation of the two-dimensional grating.

 figure: Fig. 5.

Fig. 5. The spots effect without adding a spatial filter and adding a spatial filter to the autofocus system are represented by (a) and (c) respectively. It is evident that the addition of a spatial filter after the semiconductor laser effectively enhances the uniformity of the light spots and improves its completeness. The spot effect without adding a spatial light modulator and adding a spatial light modulator to the autofocus system are depicted in (b) and (d) respectively. By incorporating a spatial light modulator, an array of light spots can be generated in the sample plane of the objective field of view and on the CCD sensor. This enables automatic focusing at different positions of the sample.

Download Full Size | PDF

Considering the reliance on autofocus spot for acquiring focus detection measurements, the shape of the autofocus spot substantially influences the precision of autofocus as well as the linearity of the autofocus curves. Because of aberrations between the autofocus system and the microscopy system in practical scenarios, autofocus spots near the focal plane undergo pronounced deformations, posing a considerable challenge to the precise retrieval of focus detection measurements, as depicted in Figs. 6(a)-(d). In an effort to improve the linearity of the autofocus curves and the rate of auto-focusing, the CCD in the autofocus system was advanced by a certain distance, effectively mitigating the detrimental influences of diffraction and aberrations on the spot's shape, as illustrated in Figs. 6(e)-(h). A comparison of the two sets of images reveals that our approach normalizes the spot shape, which in turn significantly increases the accuracy of focus detection measurement and the linearity of the autofocus curves. This technique not only refines the multi-spot autofocus technology but also optimizes conventional autofocus strategies.

 figure: Fig. 6.

Fig. 6. (a), (b), and (c) show the captures from the CCD detector positioned at the focal plane of the imaging lens with the sample defocused at -100µm, 0µm, and 100µm, respectively. (d) displays the autofocus curve for the 10× objective lens, calculated by the algorithm in this paper, with the CCD of the autofocus system positioned at the focal plane of the imaging lens. (e), (f), and (g) correspond to those taken by the CCD detector when it is located at a defocused position relative to the imaging lens, with the sample being defocused at -100µm, 0µm, and 100µm, respectively. (h) shows the CCD of the autofocus system positioned at the defocused position of the imaging lens (-40 mm), and the autofocus curve for the 10× objective lens is calculated using the same algorithm. The comparison between the two sets of images suggests that adding a defocus amount to the autofocus CCD can enhance the auto-focusing accuracy and the linearity of the autofocus curve.

Download Full Size | PDF

Figure 7 shows the experimental image captured by the CCD sensor, portraying the spots images under varying magnification objective lenses (10× and 20×) and differing degrees of defocus. Figure 8 exemplifies the utilization of Matlab for processing the array of focus detection spots. The x-axis signifies the distance of defocus, while the y-axis represents the autofocus spot diameter (measured in pixels), which is employed to construct the focus detection curve.

 figure: Fig. 7.

Fig. 7. The experimental results of the laser spots received by the CCD sensor under different magnification microscope objectives and different defocus distance values: (a) (b) (c) respectively represent the spots of the 20x objective lens at -25µm, 0µm and 25µm defocus. Images, (d) (e) (f) respectively represent the spots images of the 10x objective lens at -100µm, 0µm and 100µm defocus. Obvious radius changes of the arrayed spots can be observed from the image.

Download Full Size | PDF

 figure: Fig. 8.

Fig. 8. Experimental results for variation of spot diameters with defocus distance of different magnifications microscopes objective lens. LF means linear fitting, MO means magnification of objective.

Download Full Size | PDF

The experimental results indicate that there is a linear functional relationship between the focusing curves of objective lenses with different magnifications within their focusing range, which satisfies the requirements for autofocus of microscopy. Furthermore, the experimental focusing range and accuracy meet the demands of the industry. Among them, the 10× objective lens achieves a minimum focusing accuracy of 1µm with the focusing range of ±100µm, the 20× objective lens achieves 0.5µm with the focusing range of ±20µm, all of which meet the focus accuracy requirement of ±1/4 objective lens depth of field.

3. Analysis of different positions of light spots

The paper presents the utilization of a two-dimensional Dammann grating to generate 3 × 3 arrayed spots. The sample used in this study is a reflection mirror, which is positioned perpendicular to the optical axis. The spots at various sample positions were examined under the same magnification objective lens. The semi-circular spots captured by the CCD detector were fitted into circles using the least squares method. The resulting auto-focusing spots were labeled as c1-c9 respectively and the changes in radius of the spots were analyzed for different defocus amounts using 10× and 20× objective lenses. The results were then plotted as focus curves, with the defocus amount represented on the horizontal axis and the pixel value of the spot radius represented on the vertical axis. The curves demonstrate that each spot exhibits linear characteristics, thus meeting the auto-focusing requirements. The specific curves can be seen in Figs. 9(a)-(b).

 figure: Fig. 9.

Fig. 9. Experimental results for variation of spot diameters with defocus distance of different spots of different magnifications microscopes objective lens. (a) Focus detection curve of the 10× objective lens autofocus system. (b) Focus detection curve of the 20× objective lens autofocus system.

Download Full Size | PDF

As shown in Fig. 9, the autofocus spots at each sample location have a consistent slope of the autofocus curve, in agreement with theoretical results. However, these autofocus curves show performance variations, meaning the curves do not completely coincide. To investigate the cause of this phenomenon, we conducted simulation experiments. We constructed an aberration-free autofocus system model, and set up both a horizontal sample surface and a tilted sample surface (X tangent = 0.01, Y tangent = 0.01), and calculated the corresponding autofocus curves. Subsequently, we introduced aberrations into the model, obtaining the autofocus curves for the non-tilted sample surfaces. The experimental results are displayed in Fig. 10.

 figure: Fig. 10.

Fig. 10. (a) depicts the footprint diagram captured by the CCD sensor when simulating an autofocus system free of aberrations with the sample being in focus. Then we place the sample at a tilt (X Tan = 0.01, Y Tan = 0.01), the footprint diagram is shown in (b). (c) illustrates the footprint diagram received by the CCD sensor in a simulation of the autofocus system with aberrations. As shown in (d), the autofocus curves of multiple light spots overlap precisely in an ideal autofocus system devoid of aberrations. (e) shows that in an autofocus system free of aberrations, the curves of the spots do not overlap when the sample surface is inclined. Lastly, (f) depicts that in the system with aberrations, the curves of multiple light spots fail to align despite the sample being untilted.

Download Full Size | PDF

We ultimately identified two primary reasons for the performance disparity in autofocus curves. The first reason is the difficulty in ensuring the sample's complete perpendicularity to the optical axis. During the experiments, we employed a high-precision angular adjustment stage for meticulous alignment to minimize the impact of sample tilt on the focus detection curves. Second, aberrations in the autofocus system and the microscopy optical path may cause variations in the autofocus curves at different sample positions. Therefore, we entered all the autofocus curve data from spots at different positions into the program, ensuring that these autofocus spots do not interfere with each other, to achieve multi-position autofocus functionality.

4. Sample verification

To verify the efficacy of arrayed spots autofocus in resolving focus failure issues, we employed this system to perform a focus test on real biological analysis chip. We specifically analyzed the scenario where the periphery of sample was positioned at the microscope's field of view.

Figure 11(a) shows a microscopic image of the sample and substrate boundary of the bio-analytical chip. The bio-analytical chip consists of three regions: region A represents the sample part, region B represents the substrate part, and region C is the boundary between the two. Figures 11 (b), (c), and (d) display the light spot obtained by the autofocus CCD when the autofocus light is directed towards the sample part, substrate part, and boundary part, respectively. The spots obtained from the reflection on the sample surface appear more regular and exhibits some interference fringes. However, the light spots reflected back from the base surface are distributed in a point-like pattern rather than a regular semicircular spot due to the rough surface of the base material. Therefore, it is difficult to obtain corresponding focusing measurement values. Autofocusing failure is likely to occur when the single-point autofocus system detects light irradiation on the boundary part. In order to address this issue, we propose a method that utilizes an array of beams. As shown in Fig. 11(d), when the light is irradiated on the boundary part, it can be observed that some spots can obtain defocus values normally. This approach reduces the possibility of autofocus failure and enhances the robustness of the autofocus system.

 figure: Fig. 11.

Fig. 11. (a) The biological analysis chip used in the experiment, with region A as the sample section, region B as the substrate section, and C as the interface between the two. (b) The spot state of the focusing light in region A can be observed to have a relatively complete spot shape, with interference fringes present. (c) The spot state of the focusing light in region B can be observed to have a poor spot shape due to the scattering effect of the substrate material. (d) The spot state of the focusing light in region C, with the blue line representing the boundary, can be observed to have a normal spot in the upper half of the boundary, while the spot in the lower half becomes blurred after scattering.

Download Full Size | PDF

5. Conclusion

This study presents a novel and accurate laser-based arrayed spots autofocus system of microscope, aiming to address the autofocus failures caused by sample problems or scanning processes and improve the robustness of photoelectric-based autofocus methods for sample detection. The proposed method utilizes the feedback signal of the reflected laser spot radius, captured by the CCD sensor, to determine the defocus amount. Subsequently, it drives the motion-stage of the sample placement to move incrementally along the optical axis direction to achieve autofocus. The main reliance of the arrayed spot in this study is on the spatial light modulator inserted in the system. Furthermore, by providing a pre-defocus amount to the CCD sensor in the autofocus system, the shape of the spots can be optimized, and the influence of aberrations on the autofocus spots can be eliminated. As a result, the system accuracy is improved and the response time is shortened.

Furthermore, this study has established an optical system within the laboratory and employed a reflective mirror as a specimen to showcase the viability of the suggested arrayed spots autofocus system. The experimental findings indicate that the aforementioned autofocus system not only possesses the capability to automatically focus on samples with substantial planar errors at various focal planes, but also exhibits a wider focusing range, enhanced focusing accuracy, and improved focusing speed. Consequently, the proposed system presents an optimal resolution for a diverse array of automated detection applications in the domains of electronics, bio-medicine, and industry with high development prospects. Future work will focus on system integration and development of optical instrumentation.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. W.-Y. Hsu, C.-S. Lee, P.-J. Chen, et al., “Development of the fast astigmatic auto-focus microscope system,” Meas. Sci. Technol. 20(4), 045902 (2009). [CrossRef]  

2. L. Han, X. Wei, C. Liu, et al., “Cell transcriptomic atlas of the non-human primate Macaca fascicularis,” Nature 604(7907), 723–731 (2022). [CrossRef]  

3. B. J. Jung, H. J. Kong, B. G. Jeon, et al., “Autofocusing method using fluorescence detection for precise two-photon nanofabrication,” Opt. Express 19(23), 22659 (2011). [CrossRef]  

4. X. Zhou, P. Xiong, D. Chi, et al., “Fast autofocusing based on pixel difference with the Tanimoto coefficient between images,” Opt. Lett. 47(15), 3752–3755 (2022). [CrossRef]  

5. Z. Hua, X. Zhang, and D. Tu, “Autofocus methods based on laser illumination,” Opt. Express 31(18), 29465–29479 (2023). [CrossRef]  

6. Z. Hua, X. Zhang, D. Tu, et al., “Learning to high-performance autofocus microscopy with laser illumination,” Measurement 216, 112964 (2023). [CrossRef]  

7. S. Yazdanfar, K. B. Kenny, K. Tasimi, et al., “Simple and robust image-based autofocusing for digital microscopy,” Opt. Express 16(12), 8670–8677 (2008). [CrossRef]  

8. X. Zhang, Z. Liu, M. Jiang, et al., “Fast and accurate auto-focusing algorithm based on the combination of depth from focus and improved depth from defocus,” Opt. Express 22(25), 31237–31247 (2014). [CrossRef]  

9. D. Hu, Y. Liang, Y. Chen, et al., “Autofocusing Airy beam STED microscopy with long focal distance,” Opt. Commun. 404, 196–202 (2017). [CrossRef]  

10. C. Guo, Z. Ma, X. Guo, et al., “Fast auto-focusing search algorithm for a high-speed and high-resolution camera based on the image histogram feature function,” Appl. Opt. 57(34), F44–F49 (2018). [CrossRef]  

11. K. Du, D. Zhou, S. Zhou, et al., “High-accuracy differential autofocus system with an electrically tunable lens,” Opt. Lett. 48(11), 2789–2792 (2023). [CrossRef]  

12. K.-H. Kim, S.-Y. Lee, S. Kim, et al., “DNA microarray scanner with a DVD pick-up head,” Curr. Appl. Phys. 8(6), 687–691 (2008). [CrossRef]  

13. J. Yang, L. Qiu, W. Zhao, et al., “Laser differential reflection-confocal focal-length measurement,” Opt. Express 20(23), 26027–26036 (2012). [CrossRef]  

14. H. Wu, J. Yang, L. Qiu, et al., “Measuring the lens focal length by laser confocal technique,” in Sixth International Symposium on Precision Mechanical Measurements (SPIE), 8916, pp. 373–382 (2013).

15. K. Guo, J. Liao, Z. Bian, et al., “InstantScope: a low-cost whole slide imaging system with instant focal plane detection,” Biomed. Opt. Express 6(9), 3210–3216 (2015). [CrossRef]  

16. X. Zhang, F. Zeng, Y. Li, et al., “Improvement in focusing accuracy of DNA sequencing microscope with multi-position laser differential confocal autofocus method,” Opt. Express 26(2), 887–896 (2018). [CrossRef]  

17. A Chen, Y Sun, Y. Lei, et al., “Single-cell spatial transcriptome reveals cell-type organization in the macaque cortex,” Cell 186(17), 3726–3743 (2023). [CrossRef]  

18. S. J. Abdullah, M. M. Ratnam, and Z. Samad, “Error-based autofocus system using image feedback in a liquid-filled diaphragm lens,” Opt. Eng. 48(12), 123602 (2009). [CrossRef]  

19. H. Qu and Yang, “Optical microscopy with flexible axial capabilities using a vari-focus liquid lens,” J. Microsc. 258(3), 212–222 (2015). [CrossRef]  

20. M. Bathe-Peters, P. Annibale, and M. J. Lohse, “All-optical microscope autofocus based on an electrically tunable lens and a totally internally reflected IR laser,” (2018). [CrossRef]  

21. W.-Y. Hsu, “Automatic Compensation for Defects of Laser Reflective Patterns in Optics-Based Auto-Focusing Microscopes,” IEEE Sens. J. 20(4), 2034–2044 (2020). [CrossRef]  

22. S. H. Wang, C. J. Tay, C. Quan, et al., “Laser integrated measurement of surface roughness and micro-displacement,” Meas. Sci. Technol. 11(5), 454–458 (2000). [CrossRef]  

23. Q. Li, L. Bai, S. Xue, et al., “Autofocus system for microscope,” Opt. Eng. 41(6), 1289–1294 (2002). [CrossRef]  

24. C. -S. Liu, P. -H. Hu, Y. -H. Wang, et al., “Novel fast laser-based auto-focusing microscope,” in Sensors, (IEEE, 2010) pp. 481–485.

25. C.-S. Liu, Y.-C. Lin, and P.-H. Hu, “Design and characterization of precise laser-based autofocusing microscope with reduced geometrical fluctuations,” Microsyst. Technol. 19(11), 1717–1724 (2013). [CrossRef]  

26. C.-S. Liu, P.-H. Hu, and Y.-C. Lin, “Design and experimental validation of novel optics-based autofocusing microscope,” Appl. Phys. B 109(2), 259–268 (2012). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1.
Fig. 1. (a) Schematic diagram of the optical path for the proposed autofocus microscope system. The green and blue beams represent the beams reflected up and below the focal plane, respectively. The red beam is the beam reflected on the focal plane. δ represents the defocus amount, Δ denotes the displacement of the focusing image at the CCD detector, f1 and f2 represent the focal length of the objective lens and imaging lens, respectively. d is the spot radius of the collimated beam, and α is the semi-aperture angle of the microscope objective. (b) Image of the reflected laser beam captured from the CCD sensor in theory. Δ represents the spot radius. The blue dot and red dot signify the center of the circle and the centroid, respectively. Their corresponding coordinate are (Xc, Yc) and (Xcentroid, Ycentroid), respectively.
Fig. 2.
Fig. 2. (a) Detailed structure of proposed auto-focusing microscope features a comprehensive design that includes several key components. One of the primary modifications is the addition of a spatial filter at the initial autofocus system light source to even out the light distribution. Furthermore, a spatial light modulator is incorporated after the semicircular diaphragm to function as a light splitting device (the paper employs a two-dimensional Dammann grating to divide a beam of light into a 3 × 3 array). In addition to the structural addition, the CCD detector is also relocated towards the imaging lens which provides defocus for the autofocus detection spots. The microscope has three light paths: the blue light path for imaging, the red light-path for incident light used for focusing, and the green light path for reflected light used for focusing. The autofocus module is represented by the dotted line section. Beam splitter 1 is a semi-transparent mirror, while beam splitter 2 is a dichroic mirror. The lens is used for imaging, and the knife can also function as a semicircular diaphragm. (b) ZEMAX optical model of proposed autofocusing microscope. Red, blue, and green are the respective colors of the three beams of light that are separated by the two-dimensional grating. These beams are then reflected by the sample before reaching the CCD detector.
Fig. 3.
Fig. 3. The fig. (a), (b), and (c) depict the footprint diagram detected by the CCD sensor at defocus amounts of -50µm, 0µm, and 50µm, respectively. Meanwhile, the figures (d), (e), and (f) illustrate the footprint diagram present on the sample surface at -50µm, 0µm, and 50µm defocus. (The imaging lens has a focal length of 100 mm, and the CCD sensor is positioned 80 mm behind the imaging lens, which serves as a preset defocus amount for the focusing system.)
Fig. 4.
Fig. 4. A photograph of a laboratory-built prototype is shown. The autofocus module is indicated by the green frame line, while the remaining components consist of the infinity correction microscope parts. The red light is the incident focusing light, and the blue light is the reflected focusing light.
Fig. 5.
Fig. 5. The spots effect without adding a spatial filter and adding a spatial filter to the autofocus system are represented by (a) and (c) respectively. It is evident that the addition of a spatial filter after the semiconductor laser effectively enhances the uniformity of the light spots and improves its completeness. The spot effect without adding a spatial light modulator and adding a spatial light modulator to the autofocus system are depicted in (b) and (d) respectively. By incorporating a spatial light modulator, an array of light spots can be generated in the sample plane of the objective field of view and on the CCD sensor. This enables automatic focusing at different positions of the sample.
Fig. 6.
Fig. 6. (a), (b), and (c) show the captures from the CCD detector positioned at the focal plane of the imaging lens with the sample defocused at -100µm, 0µm, and 100µm, respectively. (d) displays the autofocus curve for the 10× objective lens, calculated by the algorithm in this paper, with the CCD of the autofocus system positioned at the focal plane of the imaging lens. (e), (f), and (g) correspond to those taken by the CCD detector when it is located at a defocused position relative to the imaging lens, with the sample being defocused at -100µm, 0µm, and 100µm, respectively. (h) shows the CCD of the autofocus system positioned at the defocused position of the imaging lens (-40 mm), and the autofocus curve for the 10× objective lens is calculated using the same algorithm. The comparison between the two sets of images suggests that adding a defocus amount to the autofocus CCD can enhance the auto-focusing accuracy and the linearity of the autofocus curve.
Fig. 7.
Fig. 7. The experimental results of the laser spots received by the CCD sensor under different magnification microscope objectives and different defocus distance values: (a) (b) (c) respectively represent the spots of the 20x objective lens at -25µm, 0µm and 25µm defocus. Images, (d) (e) (f) respectively represent the spots images of the 10x objective lens at -100µm, 0µm and 100µm defocus. Obvious radius changes of the arrayed spots can be observed from the image.
Fig. 8.
Fig. 8. Experimental results for variation of spot diameters with defocus distance of different magnifications microscopes objective lens. LF means linear fitting, MO means magnification of objective.
Fig. 9.
Fig. 9. Experimental results for variation of spot diameters with defocus distance of different spots of different magnifications microscopes objective lens. (a) Focus detection curve of the 10× objective lens autofocus system. (b) Focus detection curve of the 20× objective lens autofocus system.
Fig. 10.
Fig. 10. (a) depicts the footprint diagram captured by the CCD sensor when simulating an autofocus system free of aberrations with the sample being in focus. Then we place the sample at a tilt (X Tan = 0.01, Y Tan = 0.01), the footprint diagram is shown in (b). (c) illustrates the footprint diagram received by the CCD sensor in a simulation of the autofocus system with aberrations. As shown in (d), the autofocus curves of multiple light spots overlap precisely in an ideal autofocus system devoid of aberrations. (e) shows that in an autofocus system free of aberrations, the curves of the spots do not overlap when the sample surface is inclined. Lastly, (f) depicts that in the system with aberrations, the curves of multiple light spots fail to align despite the sample being untilted.
Fig. 11.
Fig. 11. (a) The biological analysis chip used in the experiment, with region A as the sample section, region B as the substrate section, and C as the interface between the two. (b) The spot state of the focusing light in region A can be observed to have a relatively complete spot shape, with interference fringes present. (c) The spot state of the focusing light in region B can be observed to have a poor spot shape due to the scattering effect of the substrate material. (d) The spot state of the focusing light in region C, with the blue line representing the boundary, can be observed to have a normal spot in the upper half of the boundary, while the spot in the lower half becomes blurred after scattering.

Equations (7)

Equations on this page are rendered with MathJax. Learn more.

t a n α = d / f 1
A A 1 = 2 δ t a n α
Δ = kA A 1
k = f 2 / f 1
Δ = δ 2 d f 2 f 1 2
X centroid = ( X X C ) P ij P ij
Y centroid = ( Y Y C ) P ij P ij
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.