Open Access
Add to BibSonomy
Abstract
Laser-beam absorptance in a keyhole is generally calculated using either a ray-tracing method or electrodynamic simulation, both physics-based. As such, the entire computation must be repeated when the keyhole geometry changes. In this study, a data-based deep-learning model for predicting laser-beam absorptance in full-penetration laser keyhole welding is proposed. The model uses a set of keyhole top- and bottom-aperture as inputs. From these, an artificial intelligence (AI) model is trained to predict the laser-energy absorptance value. For the training dataset, various keyhole geometries (i.e., top- and bottom-aperture shapes) are hypothetically created, upon which the ray-tracing model is employed to compute the corresponding absorptance values. An image classification model, ResNet, is employed as a learning recognizer of features to predict absorptance. For image regression, several modifications are applied to the structure. Five model depths are tested, and the optimal AI architecture is used to predict the absorptance with an R2 accuracy of 99.76% within 1.66 s for 740 keyhole shapes. Using this model, several keyhole parameters affecting the keyhole absorptance are identified.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
The complex interactions between a laser beam and a keyhole determine how and how much laser energy is absorbed in a material, which subsequently influences other physical processes, such as fluid flow, melting, vaporization, solidification, free-surface evolution, and weld quality [1,2]. Therefore, the prediction of laser-beam absorptance inside a keyhole becomes crucial in understanding the entire welding process.
For this reason, many researchers have endeavored to calculate laser-beam absorptance inside a keyhole. Ki et al. [3] studied the effect of multiple reflections on keyhole evolution in laser drilling and laser keyhole welding processes using a ray-tracing method. Amara and Bendib [4] employed a ray-tracing method for multiple reflections phenomena in their keyhole vapor-flow model by considering both simple geometries and self-consistent keyhole shapes. Cho and Na [5] implemented multiple reflections and Fresnel absorption phenomena in their laser keyhole welding model based on the volume-of-fluid method and a ray-tracing technique. Zhang et al. [6] employed a ray-tracing method to investigate a variation of bremsstrahlung absorption and power-density distribution on keyhole walls at different side-assisting gas velocities based on the assumption of a conical keyhole shape. Xu et al. [7] accounted for multiple reflections in their laser hybrid welding model based on an adaptive heat-source model. They applied a ray-tracing method on the keyhole shape estimated from the local energy balance. Han et al. [8] proposed a progressive search algorithm for a ray-tracing method and presented two new models (i.e., transmission and scattering) to simulate the laser keyhole welding process. Allen et al. [9] explored the definite positive correlation between the highly dynamic vapor-depression geometry and laser-energy absorptance using inline coherent imaging and integrating-sphere radiometry, and validated the results using ray-tracing simulations. Deng et al. [10] simulated multiple reflections phenomena inside a keyhole using the finite-difference time-domain (FDTD) algorithm for electrodynamics. In their work, the keyhole in a full-penetration welding process was reconstructed by linearly joining the top and bottom apertures, which were measured experimentally using the coaxial observation method [11]. The electrodynamic simulation is theoretically more accurate than the ray-tracing method, because all laser-beam electromagnetic-wave characteristics are naturally accounted for. Nonetheless, it is very computationally expensive.
Recently, several studies have been reported on the application of deep learning to laser keyhole welding. Knaak et al. [12] employed a convolutional neural network (CNN) for image segmentation to measure the keyhole, weld pool, weld seam, and background from the laser keyhole welding images observed using the coaxial observation method, and the process deviations were detected using a machine learning model. Shevichik et al. [13] proposed a deep learning framework to classify conduction welding, stable keyhole, unstable keyhole, blowout, and pores from the laser back reflected and acoustic emissions signals, and X-ray radiographic was employed to visualize the sub-surface behavior of the process zone. Oh and Ki [14] developed a deep learning model that predicts high-resolution optical microscopic images of the cross-sectional weld bead from the laser processing parameters.
In this article, a deep-learning model for predicting laser-beam absorptance in full-penetration laser welding is presented. This model employed a modified ResNet structure [15], which was based on a convolutional neural network [16], and predicts the laser-beam absorptance from a set of keyhole top- and bottom-aperture shapes. Unlike physics-based models (e.g., ray tracing and electrodynamic simulation), this model is a data-based model, such that absorptance can be computed instantly for other keyhole shapes. Note that the authors’ previous electrodynamic model [10] also used keyhole aperture shapes for absorptance calculation. However, full computation needed to be repeated for other keyhole shapes. In this study, the training data were prepared by applying a ray-tracing method to various keyhole geometries, and electrodynamic simulations were carried out for validation.
2. Data preparation
The schematic of this study is presented in Fig. 1. First, the shape of a 3D keyhole for full-penetration laser welding was linearly constructed from a set of top and bottom apertures (the left figure in Fig. 1(a)), and a ray-tracing method was applied to the constructed keyhole to calculate the absorptance, which served as the ground truth for the deep-learning model. To validate the accuracy of the ray-tracing method, an electrodynamic simulation using the FDTD method was carried out on several keyholes.
Fig. 1. Schematic of this study: keyhole energy absorptance was predicted from a 2D image with top and bottom apertures by using a deep-learning model.
The biggest reason why we attempted to develop this kind of predictive model is that it is extremely difficult to experimentally measure the laser absorptance in a keyhole, especially for full penetration laser welding. From our observations of actual laser keyhole welding processes, however, we found that the keyhole apertures could be modeled as ellipses and the prediction of the laser keyhole absorptance based on the top and bottom apertures could be effective in understanding the actual keyhole absorption behavior. In the authors’ previous work [10], for example, laser keyhole welding of zinc-coated and uncoated steel sheets was investigated using this idea (but with physics-based methods), and meaningful findings on keyhole absorptance were obtained. Also, although the work was done for no-penetration laser welding, Allen et al. [9] recently reported that the ray-tracing simulation results conducted on conical keyhole shapes were reasonably accurate when compared with the experimental data.
A 2D image showing the top and bottom apertures (with the laser beam located at the center of the image) was used as an input to the deep-learning model to train it to predict the corresponding absorptance value. In the input image (the left figure in Fig. 1(b)), the red and green ellipses represent the keyhole top and bottom apertures, respectively. The deep-learning model was trained to recognize essential information (i.e., aperture sizes, shapes, and relative position between apertures) for predicting the keyhole energy absorptance [10].
For the ray-tracing simulation, the material was assumed to be pure iron, and a circularly polarized continuous-wave laser beam having a Gaussian intensity profile and a wavelength of 1070 nm was used as the energy source. The beam diameter was 200 μm and focused on the top surface (the laser beam was assumed to be delivered through a 200 μm process fiber, collimated by a 160 mm collimation lens, and then focused on the surface of the specimen by a 160 mm focusing lens – thus, the focused diameter was 200 μm). In this study, the divergence of the incident laser beam was taken into account using the method used in [17], and each ray was reflected on the keyhole surface, as governed by the specular reflection law [18]:
where r, i, and n are, respectively, reflected ray-unit vector, incident ray-unit vector, and unit normal vector at a ray-surface intersection point. At each reflection point, the reflectances for the p- and s-polarizations were calculated based on the Fresnel reflection [19]:To validate the ray-tracing method used in this study, the FDTD simulation was conducted for the 15 test keyholes shown in Fig. 2. The FDTD method provides an accurate solution for beam propagation by solving Maxwell’s equations and accounting for all electromagnetic laser characteristics. Details about the FDTD [20] are omitted here for simplicity. The constants used in the FDTD simulation are given in Table 1, and the laser wavelength was enlarged (λ*) to decrease the computational cost while maintaining the absorption rate [10].
Table 1. Parameters for FDTD simulations having enlarged wavelengths at iron (λ*: enlarged wavelength, n: refractive index, κ: extinction coefficient, ε∞: dielectric constant at infinite frequency, wp: plasma frequency, and γp: damping constant).
In Fig. 3, FDTD simulation and ray-tracing results are shown (left: FDTD; right: ray tracing) with their calculated absorptance values. Figure 3 indices (a)–(d) correspond to those of Fig. 2. All 15 absorptance results are shown in Fig. 4. Given that most of the blue dots are near the black line, and the average of the absorptance difference is within 3%, the ray-tracing method appears to be sufficiently accurate to generate the datasets needed for deep learning.
Fig. 3. Comparison of beam propagation simulated by FDTD and ray tracing. In each figure, the FDTD results is shown on the left, and the ray-tracing result is shown on the right (red: incident laser beam; yellow: back reflected ray).
The dataset for the artificial-intelligence (AI) model was generated considering experimentally observed keyhole movements, in such a way that the laser-beam absorptance values in the dataset were distributed uniformly from 0% to 100%. According to the experimental results performed in the authors’ previous work [11,21], the keyhole bottom aperture was repeatedly opened and closed, causing rapid changes in location, shape, and size. The keyhole top aperture, in contrast, was always open wider than the beam size resulting in very few changes. As shown in Fig. 5, keyhole top and bottom apertures were hypothetically created. Based on the experimental observations, the size of keyhole top aperture was selected around 500 μm using a random number generator, accounting for the beam size of 200 μm. To achieve a uniform absorptance distribution (0% ∼ 100%) in the dataset, the keyhole bottom aperture was moved while reducing the size, as shown in Fig. 5. Note that the domain size was 1.01 mm × 1.12 mm, and the laser beam was located at the center of the domain for all cases.
Fig. 5. Hypothetically generated keyhole apertures. Red and yellow ellipses are the keyhole top and bottom apertures. Corresponding energy absorptance is shown at the right-bottom corner in each cell.
Both keyhole top- and bottom-aperture images were concatenated in the channel direction and input to the AI model, as presented in Fig. 1 (input size: (C, H, W) = (2, 288, 320); C, H, and W denote the number of image channels, the number of pixels in height and width directions, respectively). A total of 7418 random apertures were prepared with corresponding ground truths (i.e., absorptances obtained from the ray-tracing method) and were randomly divided into training, validation, and test sets in a 7:2:1 ratio (5194, 1484, and 740, respectively). No zero-centering or normalization was applied to the input images or ground truths. The training dataset was pre-augmented by a factor of two using a double flip method (flip x and y). Note that the augmentation was performed before entering the training loop, because it showed a ∼1%-higher accuracy than the real-time augmentation.
3. Deep residual model
As a learning-feature recognizer, the well-known ResNet structure [15] was utilized, which is based on deep convolutional layers and residual connections. Using ResNet, the layers can be deepened without gradient vanishing thanks to the skip connections applied to the entire network. Also, it has an intuitive and simple unit structure, so it is easy to apply to large models with minor modifications [22,23]. In addition, considering the possibilities of the trained model being introduced into edge devices for future welding monitoring systems, the use of the standard convolution can be more effective in terms of computational speed than the depthwise convolution [24]. Although the adopted model is not the most recent, it is an architecture that has been very successful in image classification and continues to provide the best performance [25] while using a simple and intuitive structure. We trained the vanilla ResNet architecture, but we removed the last rectified linear-unit layer in every unit block [26] to improve model performance. We also adopted a regressor with a regression loss function instead of the classification modules. Five model depths (ResNet-18, 34, 50, 101, and 152) were tested to find the optimal model depth suitable for the keyhole aperture data, and their architectures are listed in Table 2. Note that when applying deep learning, since the optimal model depth (i.e., the number of stacked layers) depends on the amount of features to be extracted, tests on model depths need to be carried out in advance for the custom datasets. The model was implemented using PyTorch, and more details about the model structure can be found in [15].
Table 2. Employed ResNet architecture for image regression from the keyhole apertures to laser-beam absorptance. Filter size and repetition number of each residual block were specified according to the depth of layers. Input aperture image size was 288, 320, and 2 in height, width, and channel directions, respectively. The output size of each layer is also presented at the bottom (output stride=32).
As presented in the table, the last classifier was replaced with a regressor that predicts a scalar value (in our case keyhole energy absorptance), from the feature maps encoded via the deep residual convolutional layers. Accordingly, the cross-entropy cost function was substituted with a mean absolute error (MAE) (J: cost function; N: number of data; y: predicted energy absorptance; A: ray tracing-simulated energy absorptance):
We also tested a mean squared error for the objective function. However, we could not achieve a better result with it.
The model was trained on a single node using quad TITAN RTX graphical processing units (GPUs) in a distributed data parallel manner. The total epochs were 250, and the initial learning rate was 10−4, which decayed by 1/10 at epochs of 50 and 150, respectively. The batch size was 100 (25 for each GPU), and the weight decay coefficient was 10−6. Adam optimizer [27] was employed. The model was trained from scratch, and, in Fig. 6, the loss curves for the training dataset (black) and validation dataset (blue) according to the epochs are presented. Only the losses for the ResNet-34 are presented, which was the optimal depth for our dataset. Loss curves for other model depths are omitted for the sake of clarity. The validation loss was minimum at epoch 218. Hence, the test dataset was evaluated using the parameters and buffers at this epoch. Note that we tried extending the training loop beyond 250 epochs. However, we could not get a better validation loss.
4. Results and discussion
In Fig. 7, the prediction result for the test dataset (740 keyhole apertures) was compared with the ground truth (y-axis: AI prediction result; x-axis: ground truth). In the left figure, the results for all five model depths are presented with different color labels, and that for depth 34, showing the best accuracy, is redrawn in the right figure.
Fig. 7. Graphical comparison between the AI prediction result (y-axis) and the ground truth (x-axis) for the test dataset (740 keyhole apertures) in terms of five model depths (left) and only the depth of 34 (right), which showed the best test accuracy among all depths (R2: 99.76%).
As shown at the left of Fig. 7, the prediction results for all model depths look fine (close to the y = x line) in all absorptance ranges. For an accurate performance check, the MAE, the R2 accuracy, and the prediction time for the models are shown in Table 3. Note that the prediction time was simply measured for purposes comparing the model depths, and the time itself may differ according to hardware and software settings. We measured the prediction time using four TITAN RTX GPUs with a single node and a batch size of 25 in each via distributed data parallelism using one subprocess for data loading. As shown in the table, the accuracy of all models exceeded 99.5%, indicating that the AI models were successfully trained to recognize key information included in the keyhole aperture images. Among the tested ResNet depths, the depth of 34 layers showed the best performance for our dataset, considering the accuracy and prediction time (99.76% and 1.66 s, respectively) for a total of 740 input keyhole apertures. The tested keyhole apertures and prediction results from the ResNet-34 are presented in Fig. 8, with prediction and ground truth annotated below each image. Among the 740 test images, randomly picked 100 images are shown, showing that absorptances were well-predicted according to the various keyhole shapes, sizes, and locations.
Fig. 8. Input keyhole apertures with absorptance prediction results from ResNet-34 and corresponding ground truths (unit: percentage). Among the 740 test images, only 100 are presented for the sake of simplicity (randomly chosen). In each figure, the prediction result and ground truth are annotated below the image reflecting prediction and ground truth.
Table 3. Mean error, accuracy, and prediction time for the test dataset according to the depth of layers.
Using the 740 ResNet-34 test results, three laser welding parameters affecting the absorptance were identified and their influence on the laser beam absorptance is shown in Fig. 9. In Fig. 9(b), the absorptance is plotted versus the keyhole front wall tilting angle θ, which is defined as ${\tan ^{ - 1}}({d_f}/t)$, where t is the specimen thickness and df is the horizontal distance between the front parts of the top and bottom apertures (See Fig. 9(a)). In Fig. 9(c), normalized bottom aperture area (${A_{b,n}} = {A_b}/{A_{beam}}$) is used as the second parameter, which is defined as the keyhole bottom aperture area (Ab) normalized by the laser beam area (Abeam). As shown, the laser-beam absorptance increases as the tilting angle increases and the bottom aperture area decreases, as a larger tilting angle leads to more reflections and a smaller bottom aperture tends to contain the beam inside the keyhole. In Fig. 9(d), the ratio of the first two parameters, θ/Ab,n, is naturally selected as the third parameter, which also shows a fairly good correlation with the absorptance. As shown in Fig. 9(b)-(d), our AI prediction results (red dots) faithfully follow the trend of the ground truth (black dots), demonstrating that the developed deep-learning model does not just recite the training data, but correctly captures the physical correlation between keyhole geometry and energy absorptance.
Fig. 9. (a) Schematic of a linearized keyhole with the dimensions used to define key parameters. Keyhole absorptance versus the (a) keyhole front wall tilting angle θ, (b) normalized bottom aperture area Ab,n, and (c) θ/Ab,n. Ground truth and AI test results of absorptance are shown as black and red dots, respectively, in each figure.
It is noteworthy that the reflection phenomena in a keyhole is very complex and affected by numerous factors. Therefore, the rather chaotic relationships together with the scattered data points shown in Fig. 9 are believed to be the results of neglecting other factors affecting the laser absorptance, such as top aperture shape and size, and the distance between the laser beam and the front keyhole wall.
5. Conclusions
Keyhole energy absorptance was efficiently predicted from keyhole top- and bottom-aperture images using a deep-learning-based image regression methodology. The major findings of this study are summarized as follows. First, various keyhole shapes were hypothetically and randomly created and used to train the AI model. For fast data generation (i.e., ground-truth absorptance), a ray-tracing method was adopted. Second, a ResNet structure was employed as the learning feature recognizer from the input keyhole aperture images. A depth of 34 layers was found to be the most efficient with the test accuracy and prediction time being 99.76% and 1.66 s, respectively, for 740 input keyhole apertures. Finally, from the simulation results, keyhole front wall tilting angle, normalized bottom aperture area and the ratio of the former and the latter were identified as the parameters affecting the absorptance.
Funding
National Research Foundation of Korea (2019R1A2C2089114).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. H. Ki, P. S. Mohanty, and J. Mazumder, “Modeling of laser keyhole welding: Part I. Mathematical modeling, numerical methodology, role of recoil pressure, multiple reflections, and free surface evolution,” Metall. Mater. Trans. A 33(6), 1817–1830 (2002). [CrossRef]
2. H. Ki, P. S. Mohanty, and J. Mazumder, “Modeling of laser keyhole welding: Part II. Simulation of keyhole evolution, velocity, temperature profile, and experimental verification,” Metall. Mater. Trans. A 33(6), 1831–1842 (2002). [CrossRef]
3. H. Ki, P. S. Mohanty, and J. Mazumder, “Multiple Reflection and its Influence on Keyhole Evolution,” J. Laser Appl. 14(1), 39–45 (2002). [CrossRef]
4. E. H. Amara and A. Bendib, “Modelling of vapour flow in deep penetration laser welding,” J. Phys. D: Appl. Phys. 35(3), 272–280 (2002). [CrossRef]
5. J.-H. Cho and S.-J. Na, “Implementation of real-time multiple reflection and Fresnel absorption of laser beam in keyhole,” J. Phys. D: Appl. Phys. 39(24), 5372–5378 (2006). [CrossRef]
6. L. Zhang, J. Zhang, and S. Gong, “Mechanism study on the effects of side assisting gas velocity during CO 2 laser welding process,” J. Appl. Phys. 106(2), 024912 (2009). [CrossRef]
7. G. Xu, C. S. Wu, G. Qin, X. Wang, and S. Lin, “Adaptive volumetric heat source models for laser beam and laser+ pulsed GMAW hybrid welding processes,” Int. J. Adv. Manuf. Technol. 57(1-4), 245–255 (2011). [CrossRef]
8. S.-W. Han, J. Ahn, and S.-J. Na, “A study on ray tracing method for CFD simulations of laser keyhole welding: progressive search method,” Weld. World 60(2), 247–258 (2016). [CrossRef]
9. T. R. Allen, W. Huang, J. R. Tanner, W. Tan, J. M. Fraser, and B. J. Simonds, “Energy-Coupling mechanisms revealed through simultaneous keyhole depth and absorptance measurements during laser-metal processing,” Phys. Rev. Appl. 13(6), 064070 (2020). [CrossRef]
10. C. Deng, J. Kim, S. Oh, and H. Ki, “Electrodynamic simulation of energy absorption in laser keyhole welding of zinc-coated and uncoated steel sheets,” J. Mater. Process. Technol. 231, 412–421 (2016). [CrossRef]
11. J. Kim, S. Oh, and H. Ki, “A study of keyhole geometry in laser welding of zinc-coated and uncoated steels using a coaxial observation method,” J. Mater. Process. Technol. 225, 451–462 (2015). [CrossRef]
12. C. Knaak, G. Kolter, F. Schulze, M. Kröger, P. Abels, M. E. Zelinski, T. M. Taha, J. Howe, A. A. Awwal, and K. M. Iftekharuddin, “Deep learning-based semantic segmentation for in-process monitoring in laser welding applications,” presented at the Applications of Machine Learning 2019.
13. S. Shevchik, T. Le-Quang, B. Meylan, F. V. Farahani, M. P. Olbinado, A. Rack, G. Masinelli, C. Leinenbach, and K. Wasmer, “Supervised deep learning for real-time quality monitoring of laser welding with X-ray radiographic guidance,” Sci. Rep. 10(1), 3389 (2020). [CrossRef]
14. S. Oh and H. Ki, “Cross-Section Bead Image Prediction in Laser Keyhole Welding of AISI 1020 Steel Using Deep Learning Architectures,” IEEE Access 8, 73359–73372 (2020). [CrossRef]
15. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition (2016), pp. 770–778.
16. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE 86(11), 2278–2324 (1998). [CrossRef]
17. M. Sohail, S.-W. Han, S.-J. Na, A. Gumenyuk, and M. Rethmeier, “Numerical investigation of energy input characteristics for high-power fiber laser welding at different positions,” Int. J. Adv. Manuf. Technol. 80(5-8), 931–946 (2015). [CrossRef]
18. H. Ki, P. S. Mohanty, and J. Mazumder, “Multiple reflection and its influence on keyhole evolution,” in International Congress on Applications of Lasers & Electro-Optics (Laser Institute of America2001), pp. 933–942.
19. W. M. Steen and J. Mazumder, “Laser material processing,” Springer-Verlag, London4th ed. (2010).
20. Y. Kane, “Numerical solution of initial boundary value problems involving maxwell's equations in isotropic media,” IEEE Trans. Antennas Propag. 14(3), 302–307 (1966). [CrossRef]
21. J. Kim, S. Oh, and H. Ki, “Effect of keyhole geometry and dynamics in zero-gap laser welding of zinc-coated steel sheets,” J. Mater. Process. Technol. 232, 131–141 (2016). [CrossRef]
22. S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated Residual Transformations for Deep Neural Networks,” (2016), p. arXiv:1611.05431.
23. S. Zagoruyko and N. Komodakis, “Wide Residual Networks,” (2016), p. arXiv:1605.07146.
24. A. Gholami, K. Kwon, B. Wu, Z. Tai, X. Yue, P. Jin, S. Zhao, and K. Keutzer, “SqueezeNext: Hardware-Aware Neural Network Design,” (2018), p. arXiv:1803.10615.
25. A. Kolesnikov, L. Beyer, X. Zhai, J. Puigcerver, J. Yung, S. Gelly, and N. Houlsby, “Big transfer (BiT): General visual representation learning,” arXiv preprint arXiv:1912.11370 (2019).
26. S. Gross and M. Wilber, “Training and investigating residual nets,” http://torch.ch/blog/2016/02/04/resnets.html (2016).
27. D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).
