Laser cladding powder flow field detection system based on ISR optimization algorithm

3.1. Image analysis division and improvement of ISR algorithm

Coaxial powder feeding is a commonly used method in LC, which has unique characteristics compared to other methods such as side axis powder feeding. In coaxial powder feeding technology, the laser beam and powder flow are coupled together and output from the same axis, and then they interact and enter the molten pool. This method involves the three-phase flow process of gas phase, solid phase, and liquid phase, as shown in Fig. 1.

Fig. 1Diagrams of coaxial powder feeding cladding experiment and acquisition system

The experimental process in Fig. 1 is carried out from the right to the left. The powder is fed onto the base platform by a powder feeder and heated by a fiber laser light source. High speed cameras are used for image acquisition and filters are taken to filter out environmental light interference. Finally, it is processed and collected by image processing software. To make the image of PFF clearer, ISR algorithm is introduced to improve LC detection, and low resolution in painting is repaired or reconstructed into high resolution image through software processing, and details are optimized. This study uses the Super Resolution CNN (SRCNN) model in learning methods to process images and improve it. Fig. 2 shows the structure of SRCNN.

Fig. 2Structure diagram of the SRCNN network model

The SRCNN model in Fig. 2 consists of three convolutional layers. The first layer extracts the features of the image and represents the low resolution extracted features as high-dimensional vectors, as shown in Eq. (1):

where, $Y$ represents the collected low resolution image. $W_1$ and $B_1$ represent filters and biases, respectively. The second convolution kernel is used to map the high-dimensional vector input at the first level to another high-dimensional vector through nonlinear mapping. The calculation process is Eq. (2):

where, $F_2$ is the output high-dimensional resolution image block. $W_2$ and $B_2$ are filters and biases. The function of the third convolutional kernel is to reconstruct high-resolution images, as shown in Eq. (3):

where, $F_3$ is the reconstructed high-dimensional resolution image block, while $W_3$ and $B_3$ are both filters and biases. In this study, cavity convolution is introduced to replace the first layer of convolution, so as to increase the size of Receptive field under the same computing resources. The calculation formula is Eq. (4):

where, $i$ is the input dimension, $p$ is padding, $k$ is the convolutional kernel, $d$ is the extension factor, and $s$ is the step size. Cavity convolution can lift the receptive field without increasing the training parameters. The method of introducing cavity convolution in SRCNN is listed in Fig. 3.

Fig. 3The introduction of cavity convolution

Fig. 3 first performs feature extraction on the input image, and then fills a hole with a value of 0 between adjacent values in the convolutional kernel. The white grid in Fig. 3 represents the input image, the blue grid represents the convolution kernel, and the red grid represents the pixels being processed. In the training process of SRCNN, the model evaluation loss function between the input and the original image is Eq. (5):

where, $MSE$ is the Mean squared error loss function. $\left(m,n\right)$ is the image size. $X\left(i,j\right)$ and $Y\left(i,j\right)$ are the pixel value of original and output image. The loss function is improved by introducing structural similarity and PSNR, which is shown in Eq. (6):

where, $\alpha ,\beta$ are the weight parameter for structural similarity and PSNR, respectively. $MAX$ is the max-grayscale value. $u_x$ and $u_y$ refer to the mean values. ${\delta }_x^2$ and ${\delta }_y^2$ represent the variances, and ${\delta }_{xy}$ is the covariance of Figures $X$ and $Y$. $c_1$ and $c_2$ are constants.

3.2. Optimization processing of powder flow transport image based on combined algorithm

During the coaxial powder feeding LC process, the powder will form defects such as sticky powder and unmelted powder on the melt channel, melt pool, and substrate. To study powder flow and reflect the quality of cladding, it is necessary to analyze the dynamic behavior and transport morphology characteristics of the PFF. Due to the large number of powder particles, to facilitate research and statistics, this paper proposes an image processing algorithm, as displayed in Fig. 4.

Fig. 4Route of PFF image particle processing

The route in Fig. 4 includes two stages: the original image noise background removal and the focal plane powder particle removal. In the first stage, the image is grayed out and the Canny edge detection algorithm is used to obtain the edge information of the object. Corrosion and dilation operations are performed, and the processing results are obtained through contour detection and logical operations. In the second stage, on the ground of the first stage results, the image is grayed out, contour detection is performed, and calculations are performed. The threshold difference is adjusted to filter the required particles, and the processed image is ultimately obtained. Target tracking uses Distance Precision (DP) to represent the distance between the tracking result and the actual box, with smaller values indicating better tracking performance. The calculation formula is Eq. (7):

where, $p_{i,j}$ is the center position of the target tracking result box for the $j$-th image in the $i$-th video. $p_{i,j}^{\mathrm{\text{'}}}$ is the center position of the target real box for the $j$-th image in the $i$-video. $M_i$ is the quantity of video frames for $i$ video. $N$ is the gross of tracked videos. In this manuscript, DaSiamRPN algorithm and Kalman filter algorithm are combined to track discontinuous powder particles in the PFF. This combination can improve the tracking of particles in the PFF, overcome the problems of large number of particles, high similarity and unclear characteristics, and improve the tracking performance. Fig. 5 shows its network framework. As shown in Fig. 5, the data extracted by the DaSiamRPN algorithm will be input into a Kalman filter, which is more suitable for processing data with high similarity. Finally, the final result will be output by the Kalman filter.

Fig. 5Network structure of SiamRPN

SimaRPN in Fig. 5 is an algorithm used for target tracking. The DaSiamRPN algorithm combines non maximum suppression and the distractor aware function to improve tracking accuracy. The use of DaSiamRPN in PFF images can better track powder particles. In addition, Kalman filter is a Linear filter, which can be used to predict the position of powder particles, especially in the case of disappearance and appearance of powder particles. Eq. (8) is the discrete model of the Kalman filter algorithm:

where, $x_k$ and $x_{k+1}$ mean the state variables at time $k$ and $k+1$. $z_k$ means the observed variable. $u_k$ is the control matrix. $W_k$ represents dynamic noise. $V_k$ is the observation noise. $A$ and $B$ are State-transition matrix and state control matrix. $H$ is the observation matrix. Since the Motor system of powder particles has no control input, the equation of state of the final system is Eq. (9):

where, $w_k$ is the measurement noise. The motion of most powder particles within the camera’s field of view can be regarded as a uniform linear motion, and the State-transition matrix $A$ is Eq. (10):

where, $\mathrm{\Delta }t$ is the time interval. The system only needs to take the position of the state variable to observe the variables, and its observation matrix is Eq. (11):

During the cladding process, powder is ejected from the nozzle and transported to the molten pool, while some powder particles also sputter from the cladding molten pool area. Therefore, based on the distribution area of the PFF image, the powder transfer ratio index can be defined as Eq. (12):

where, $N_S$ and $N_P$ respectively represent the number of powder particles counted in the powder splashing area and powder flow conveying area in the Figure. The powder utilization rate significantly affects both the effectiveness and quality of the cladding process. This rate is determined by the ratio between the added mass of the melt after cladding and the quality of the cladding powder used. The powder utilization rate calculated based on the weighing method in this paper is Eq. (13):

where, $W_{C1}$ is the mass of the substrate before cladding, $W_{C2}$ is the common mass of the substrate and cladding layer after cladding, and $W_P$ is the mass of the powder material used during the cladding process. The powder transfer ratio and powder utilization rate have the same trend of variation influenced by PP. The linear fitting between the two satisfies Eq. (14):

where, ${\eta }_{wei}$ is the fitting function, and 1.1144 and 17.24 are empirical parameters. During the LC process, powder particles splatter and adhere to the cladding layer’s cover, affecting surface roughness. The adhesion of powder particles depends on the collision elasticity and the adhesion force of the melt. Comparing the ratio of these two forces can determine whether there is powder sticking phenomenon, as shown in Eq. (15):

where, ${\sigma }_m$ is the surface tension, $K_P$ is the performance related constant of the powder material, $r_p$ is the size of the powder particle radius, and $v_p$ is the motion speed of the powder particles.

4.1. Powder particle recognition detection and statistics

The dataset used in this experiment was obtained in a metal smelting and processing workshop. The research team arranged high-speed cameras in the workshop to continuously capture images of powder movement during the work process for multiple days. Finally, a total of 28665 images were obtained, with a size of 14.26 MB and an image format of jpg. The dataset was divided into a test set and a training set in a 7:3 ratio based on the set aside method. Training sets were used to train the model. The training set data would be input into the constructed model using Python language and various comparison models to complete the training of all models. Then, the original PFF images from the test set would be input into the network for processing, resulting in a higher resolution PFF image for subsequent tracking progress and image quality comparison analysis. Firstly, the research team used the K-Cross Validation method to validate the robustness and stability of the design algorithm. The test results showed that if the sample was divided into 10 points, the ratio of the standard deviation of the loss function calculated for each sample to the average loss function was 0.26 %. This indicated that the designed method had good robustness and stability, and could be used for subsequent experiments and statistics. Fig. 6 shows the results of the training.

Fig. 6Comparison of PFF image evaluation results

According to the results in Fig. 6, in contrast with the original SRCNN, the improved network showed a significant improvement in the Structural Similarity Index (SSIM) after training more than 50 times, with a stable SSIM value of 0.94. The SSIM values of the models without improved algorithms and without the use of ISR algorithms were stable at 0.83 and 0.74, respectively. This indicates that the reconstructed high-resolution PFF image has not been distorted after magnification, and the details of the powder particles have been enhanced.

To understand the improvement effect of the improved algorithm designed in this study, ablation experiments are now being conducted, and the statistical results are shown in Table 1. In Table 1, “Imp-ISR_SRCNN” represents ISRs constructed based on traditional SRCNN. "Imp-ISR_daSiamRPN” represents ISRs constructed without DaSiamRPN module. “Imp-ISR_Kalman filtering" represents ISRs constructed without Kalman filter conditions. Table 1 shows that the final improved ISR algorithm designed in this study performs the best in SSIM, Peak signal-to-noise ratio (PSNR), and loss function value indicators after training. This indicates that the improvements made in this study are necessary and effective.

After processing the image of the powder flow area, the pixel distribution maps for various grayscale values are obtained in Fig. 7. The legends with white, yellow, and blue filling colors in Fig. 7(b) represent powders with grayscale values ranging from 0 to 75, 75 to 155, and 155 to 255 in the powder flow field.

Fig. 7Histogram distribution of PFF image

a) Histogram of number of pixels

b) Powder flow field simulation diagram

Fig. 7(a) shows the histogram of the pixel numbers, which shows that the grayscale values of the powder flow conveying area and the powder flow splashing area are mainly concentrated around 75 to 155. The grayscale values of 75 to 255 account for a relatively small proportion. Fig. 7(b) is a simulation diagram of the distribution of particle pixels with different grayscale values in space. Based on the manually selected powder particle analysis results, powder particles in different states can be distinguished by differences in grayscale values and pixel areas. For example, the grayscale threshold is represented by white squares at 75. A grayscale threshold between 75 and 155 and a pixel area between 3 and 20 are yellow squares. A grayscale threshold above 155 is a blue square. Yellow squares are most widely distributed in space. To facilitate accurate statistics of powder particle velocity, hierarchical statistics are performed, followed by velocity detection of powder particles in the powder flow conveying area. Fig. 8 shows the results.

Fig. 8Results of velocity distribution in powder splash field

a) Results of velocity distribution in powder splash field

b) DP value in each group

From the results in Fig. 8(a), the powder particle velocity distribution in the powder flow conveying area of different models is relatively consistent, and the velocity changes of each group are small. The traditional ISR model is generally lower in terms of statistical pixel particles compared to the improved ISR model, and the two algorithm models have the highest number of particle velocities in the 2-6 m/s range. This is because in this area, the powder particles are affected by both the protective gas and gravity, and the protective gas parameters remain unchanged. The horizontal axis in Fig. 8 represents the image data of metal smelting powder extracted at different times, which are divided into A, B, C, D, E, F, G, H, I, and J numbers according to the data generation time. From the DP values of different algorithm models in eight datasets A to J in Fig. 8(b), the DP value of the optimized ISR is basically around 3.5. The traditional ISR algorithm is higher than the improved algorithm, with a DP value around 5. Fig. 9 shows the angle detection results of the powder splash zone.

Fig. 9Results of angle detection in powder splash field

From Fig. 9, the majority of powder particles in the powder splash zone have a relatively small flight angle, concentrated at approximately 30 degrees, as quantified by the improved ISR measurement. This is mainly due to the collision between powder particles and the rebound effect after collision with the solid substrate. However, the majority of particles identified by traditional ISR exhibit larger angles, largely concentrated at around 50 degrees. This is due to the fact that particles with small high-speed deflection angles tend to fall outside the molten pool and thus are not easily captured by the model. Through the detection data of speed and angle, it is discovered that the slower flying powder particles have lower rebound height, smaller angle, and closer distance. Faster splashing powder particles have a larger rebound angle and a longer distance.

4.2. Analysis of the influence of dynamic characteristics of PFF on the quality of cladding

Based on the aforementioned research and analysis, the correlation between PFF morphology characterization indicators in high-speed images and the cladding layer's quality is investigated. In powder feeding LC, the powder utilization rate represents the proportion of powder entering the molten pool in the total powder amount. Table 2 shows the relationship between powder transfer ratio and powder utilization rate in this experiment. As shown in Table 2, the ${\eta }_{img}$ and ${\eta }_{wei}$ are calculated from the basic data using corresponding calculation formulas, and the reliability of the calculation results is verified through expert group inspection. This study invites 10 domestic metal smelting experts to participate in the research, assisting in verifying the experimental steps and results. The verification results show that all experts believe that the calculation results in Table 2 are correct.

From Table 2, as the laser power rises, the powder utilization rate also increases. Through analysis, it can be concluded that the influence of powder transfer ratio and powder utilization rate on laser power is consistent. However, if the powder feeding rate outstrips the threshold, the powder utilization rate will decrease. Excessive powder inhibits complete heating and melting of the powder flow by the laser, consequently leading to reduced utilization. In addition, under the same laser power and powder feeding rate, changing the scanning rate will slightly reduce the effective utilization rate of the powder. Therefore, powder transfer ratio based on powder image detection can replace real-time monitoring of powder utilization rate, providing a basis for real-time control of the forming process. Fig. 10 shows the influence of different laser values on the cladding layer’s flatness.

Fig. 10(a) is the statistical results of traditional ISR, where the percentage of influence of laser power is 44.6 %, scanning rate is 23.1 %, and powder transfer is 32.3 %. Fig. 10(b) shows the statistical results of the improved ISR, with a scanning rate impact percentage of 34.2 %, laser power of 41.1 %, and powder transfer of 24.7 %. The variation range of both is relatively small, indicating that the laser power has the most outstanding impact on the roughness of the cladding layer, while the powder feeding rate has the smallest impact. It should be noted that although the powder feeding rate has the smallest impact, the splashed powder particles can also have an impact on the surface roughness of the cladding layer. Fig. 11 shows the distribution data of sticking powder formed by splashing powder particles at different speeds and angles. To reduce the workload of data statistics, only 1 % of all powder particles are randomly sampled for this calculation.

Fig. 10Percentage effect of different laser parameters on the flatness of the molten layer

a) Traditional ISR algorithm model detection results

b) Improved ISR algorithm model detection results

According to the results in Fig. 11, the velocity and angle of powder particles have an impact on the number of adhered particles. In Fig. 11(a), as the speed of powder particles increases, the rapid rebound increases the proportion of splashed powder, resulting in a decrease in the number of adhered powder particles. In Fig. 11(b), as the powder angle increases, the number of collisions between most powder particles and the substrate decreases, causing a deduction in the particle numbers that are prone to adhesion. An expert group is invited to validate the data in Fig. 11. The validation results indicate that the expert group members believe that the overall trend displayed in Fig. 11 is correct and in line with industry common sense. Although the overall trend of ISR before and after improvement is consistent, traditional ISR is about 10 % less accurate in counting high-speed and high angle particles compared to improved ISR, so it is not as accurate as the improved algorithm. The improved ISR algorithm utilizes both the DaSiamRPN algorithm and the Kalman filter algorithm to process data, resulting in significant data compression and extraction. This approach allows for the extraction of correlation relationships in high similarity data. This is also the reason and advantage of combining the DaSiamRPN algorithm and the Kalman filter algorithm for powder particle data optimization in this study. In short, splashed powder particles with different speeds and angles will fall at different positions, while splashed powder particles with low speeds and angles are more likely to form surface adhesion.

Fig. 11Sticky powder distribution formed by sputtering powder particles at different speeds and angles

a) Particle adhesion distribution at different speed

b) Particle adhesion distribution at different angle