A novel wind turbine fault diagnosis method based on improved TFMST and DSC-CNN-GRU model

2.1. Improved TFMST

The fault signals of wind turbines are typically composed of various components such as harmonics, pulses, and frequency modulation. Commonly used TFA methods, such as STFT, WT, and Hilbert-Huang Transform (HHT) [34], are affected by the Heisenberg uncertainty principle, which requires a trade-off between time and frequency resolution. Attempting to accurately capture the instantaneous characteristics of the signal at a particular moment (i.e., increasing time resolution) may lead to a decrease in frequency resolution, and vice versa. The quantification of TF features requires the Instantaneous Frequency (IF) and Group Delay (GD) parameters. However, TF features are not completely independent time and frequency components, so a multi-compression approach is needed to effectively fuse the time and frequency information. TFMST method effectively integrates temporal and spectral information to generate a more concentrated TF feature representation. However, this method still exhibits certain limitations, as some fault features within the time-frequency representation may be lost. For instance, the original TFMST approach suffers from fixed threshold values, static window function parameters, and overly stringent curvature criteria. Building upon this foundation, the present study introduces enhancements to the TFMST algorithm addressing these aspects: thresholding strategy, window function parameters, curvature criteria, and numerical stability.

First, in the original algorithm, the Gaussian window parameter ($\sigma =$ 0.32) resulted in deficient temporal resolution. For certain signals, this made it difficult to capture rapid transients within fault signals, adversely impacting feature extraction accuracy. To better capture transient faults and enhance sensitivity to weak fault signatures, the Gaussian window parameter was adjusted to 0.28. This modification controls the trade-off between temporal and frequency resolution:

where $t$ denotes the time coordinate.

In the original algorithm, the instantaneous frequency is estimated by directly employing division. Its instantaneous frequency estimate is given by:

where $\omega$ represents the frequency coordinate, $S_h$ denotes the STFT of the signal, $S_{dh}(t,\omega )$ signifies the STFT of the signal convolved with the derivative of the window function, and $va$ is the scaling factor. Numerical instability can occur when the value $S_h$ is very small. Consequently, building upon the original algorithm, we incorporated an epsilon (eps) safeguard to prevent division-by-zero errors and ensure numerical stability. The modified instantaneous frequency estimate incorporating epsilon is expressed as:

where $\epsilon$ denotes the machine epsilon, preventing division by zero.

Furthermore, the original algorithm utilized a fixed threshold, setting points in the time-frequency matrix with magnitudes below 0.2 times the maximum value to zero. For diverse signals, this approach fails to adapt to variations in statistical characteristics. Concurrently, a fixed threshold can excessively suppress weak fault features and exhibits sensitivity to outliers, where a single large magnitude value could disproportionately influence the entire threshold level. To enhance the robustness of threshold setting and achieve a better balance between feature preservation and noise suppression, we implemented an adaptive thresholding scheme combining the median and maximum values. The threshold function from the original algorithm is defined as:

where ${\alpha }_1\text{=}$$\text{0.2}$, $M=\max \left|S(t,\omega )\right|$.

The improved threshold function incorporates multiple scaling thresholds, including the median threshold, maximum threshold, and base threshold. By leveraging the signal’s global statistical characteristics, the threshold is determined dynamically to adapt to the energy distribution of diverse signals.

The median threshold is defined as:

where ${\alpha }_{max}=$ 0.25. The median exhibits inherent robustness to outliers, representing the signal’s typical magnitude. The coefficient ${\alpha }_m=$ 0.15 is configured to preserve components exceeding a specified proportion of this characteristic amplitude.

The maximum threshold:

where ${\alpha }_{max}=$ 0.25, this constraint prevents the threshold value from becoming excessively high, thereby avoiding the suppression of significant signal components while maintaining effective noise reduction.

The base threshold:

where ${\alpha }_b=$ 0.1, this configuration ensures the threshold does not drop excessively low, thereby preventing the retention of extraneous noise components.

Adaptive threshold synthesis:

Improved Thresholding scheme:

where $I$ denotes the indicator function.

Thus, when signal quality is sufficiently high, $T_{median}\approx T_{base}$. Under conditions of anomalous noise, $T_{max}$ provides upper-bound protection.

The curvature criterion serves to distinguish between signal-dominant components and noise-dominant components. It operates based on derivatives of instantaneous frequency and group delay. The curvature ratio, which characterizes local geometric properties in the time-frequency distribution, is defined as:

where $\partial {\widehat{\omega }}_1/\partial t$ represents the time derivative of instantaneous frequency, and $\partial {\widehat{t}}_1/\partial \omega$ denotes the frequency derivative of group delay.

The original algorithm employs a strict curvature condition, with the feature mask characterized by:

The application of stringent curvature criteria may result in excessive misclassification of valuable fault features as noise, particularly diminishing sensitivity to faults exhibiting slight frequency modulation. To enhance fault feature detection sensitivity and preserve discriminative time-frequency patterns, we implement relaxed curvature conditions with epsilon-based numerical safeguarding. Adjusting the curvature ratio threshold from 1.0 to 1.2 and reducing the group delay derivative constraint from 0.2 to 0.15 enables retention of signal components with mild nonstationarity, thereby improving fault feature detection rates. The relaxed feature mask is defined as:

where $r_{protected}(t,\omega )=\frac{\partial {\widehat{\omega }}_1/\partial t}{\mathit{max}\left(\left|\frac{\partial \widehat{t}}{\partial \omega }\right|,\epsilon \right)}$.

Improved signal separation:

In the final energy allocation scheme, the original algorithm assigns equal weights to both the frequency directional synchrosqueezing result and the time directional synchrosqueezing result. By reducing the weight assigned to the time directional result to 0.9, we effectively diminish noise introduced through temporal reallocation, yielding enhanced output clarity. The modified weight configuration is implemented as follows:

where component $S_1^2$ satisfies the curvature condition and exhibits high signal-to-noise ratio. Component $S_2^2$potentially contains elevated noise levels; therefore, its assigned weight is reduced.

2.2. A depthwise separable convolution-CNN-GRU fusion network

CNN have been widely applied in the field of fault diagnosis, particularly for processing complex signals, images, and time-series data. Owing to their powerful capability for automatic feature extraction, CNN have demonstrated superior performance in scenarios where traditional methods encounter difficulties. Within the CNN architecture, convolutional layers, pooling layers, and fully connected layers constitute the three primary layer types [35, 36]. The Gated Recurrent Unit (GRU), an advanced variant of the Recurrent Neural Network (RNN), was introduced by Kyunghyun Cho et al. [37]. in 2014. Leveraging gating mechanisms, the GRU selectively updates its hidden state to capture essential information while discarding irrelevant details [38]. Compared to the Long Short-Term Memory (LSTM) [39] network, the GRU possesses a simpler structure and exhibits lower computational complexity. Depthwise separable convolution (DSConv)] [40] is an improved convolutional method that decomposes a standard convolution into two separate operations: depthwise convolution (DWConv) and pointwise convolution (PWConv). The specific process is illustrated in Fig. 1.

Fig. 1Depthwise separable convolution process

The distinction between DSConv and standard convolution lies in the fact that depthwise convolution employs single-channel kernels. It convolves each input channel separately, producing an output feature map with the same number of channels as the input. Subsequently, pointwise convolution (PWConv) with a kernel size of 1×1 is applied to increase dimensionality by aggregating information across all channels.

Assume the DWConv has a kernel size of $D_K\times D_K\times 1$, a number of kernels $M$, and a parameter count of $D_K\times D_K\times M$. Assume the PWConv has a kernel size of $1\times 1\times M$, a number of kernels $N$, and a parameter count of $M\times N$. Therefore, the total parameter count for the DSConv is $D_K\times D_K\times M+M\times N$.

Whereas a standard convolution has a parameter count of $D_K\times D_K\times M\times N$, the parameter ratio between DSConv and standard convolution is $\frac{1}{N}+\frac{1}{D_K^2}$. DSConv first performs DWConv independently on each input channel, then merges all channels into the output feature map via PWConv. This design achieves reduced computational cost and improved operational efficiency.

Based on relevant theoretical analysis and experimental validation, we propose a Dual-input Deep Separable Convolution-CNN-GRU Fusion Network (DSC-CNN-GRU) model. The proposed model primarily consists of two branches. Branch 1 is composed of standard convolutions and DSConv. The input data first undergoes preliminary feature extraction through a standard convolutional layer, followed by deeper feature extraction via two separable convolution modules. The function of Branch 1 is to extract features from image sample data. Branch 2 integrates a CNN model with a GRU model, where the CNN part adopts a lightweight design. This branch achieves feature extraction from one-dimensional data. Fig. 2 illustrates the specific structure of the proposed model.

Fig. 2The structure of depthwise separable convolution-CNN-GRU fusion network

The proposed model’s Branch 1 primarily extracts features from the TFMST time-frequency diagrams. In this branch, the input first undergoes a convolution operation with a 5×5 kernel size, followed by feature extraction through two DSC blocks. Within these DSC blocks, the DWConv layers use kernel sizes of 3×3 and 5×5 respectively, and the PWConv layers use a kernel size of 1×1. Finally, the features are flattened via a fully connected layer and a flatten layer, with the purpose of fusing them with the features from Branch 2. Branch 2 primarily extracts features from the raw signal. This branch mainly consists of four convolutional blocks. The second convolutional block uses a kernel size of 1×1, while the other three blocks use a kernel size of 1×3. LeakyReLU is used as an activation function throughout Branch 2. Furthermore, max pooling is employed throughout the proposed model to reduce the dimensionality of the feature maps.

The network parameters and hyperparameter settings of the DSC-CNN-GRU model are shown in Table 1. In Branch 1, a 5×5 convolution kernel is first employed for initial feature extraction, with a stride of 2 to achieve downsampling. This rapidly reduces the spatial dimensionality and extracts fundamental visual features. The two DSC blocks decompose standard convolution into depthwise convolution and pointwise convolution. DSC Block 1 uses a 3×3 convolution kernel, significantly reducing computational load while maintaining feature extraction capability. DSC Block 2 employs a larger receptive field (5×5) to capture more complex feature patterns. In Branch 2, 1×3 convolution kernels are first used, specifically designed for processing temporal data. They perform local feature extraction along the temporal dimension while preserving channel integrity. This is followed by a 1×1 convolution for dimensionality reduction, then another 1×3 convolution for feature extraction, and finally a step to restore dimensions. This design significantly reduces the number of parameters and enhances nonlinear expressive power. Subsequently, 128 GRU units are utilized for temporal modeling to capture long-range dependencies within the sequential data. Only the output from the final timestep is taken, yielding a compressed representation of the entire sequence. Following feature extraction in both Branch 1 and Branch 2, the 128-dimensional feature vector from the Light-CNN branch (Branch 1) and the 128-dimensional feature vector from and the 128-dimensional feature vector from the GRU branch (Branch 2) undergo element-wise summation. This achieves deep integration of the visual and temporal features.

The total number of learnable parameters in the final proposed model is 2.5M. Compared to classical neural network models such as AlexNet [41] (approximately 60M parameters) and ResNet-18 [42] (approximately 11.7M parameters), the proposed model achieves a substantial reduction in parameters. This lightweight architecture significantly enhances training efficiency, paving the way for future applications in fault diagnosis.

2.3. The proposed fault diagnosis method

Building upon the methods and models described above, the enhanced TFMST time-frequency data and the raw fault data are utilized as two distinct fault datasets. These dual-modal inputs are simultaneously fed into the DSC-CNN-GRU hybrid neural network model to perform fault diagnosis. the diagnostic workflow is illustrated in Fig. 3.

Fig. 3Fault diagnosis procedure based on the DSC-CNN-GRU model

The specific procedural steps are as follows:

Step 1: Data Processing. Preprocess the raw fault data to facilitate subsequent analysis.

Step 2: Data Transformation. Address limitations in the original TFMST method including fixed thresholds, fixed window function parameters, and strict curvature criteria by employing the improved TFMST method. This generates 2D time-frequency diagrams with enhanced energy concentration and richer feature representation. Process the raw data through two distinct methods to obtain two different data types.

Step 3: Sample Division. Includes two data types corresponding to two sample sets. The first sample set contains improved TFMST 2D time-frequency diagrams. The second sample set contains raw fault data. Divide both sample sets into training and testing subsets according to a specified ratio.

Step 4: Model Construction and Hyperparameter Configuration. Construct the DSC-CNN-GRU model and configure its hyperparameters. This model comprises two pathways to achieve feature fusion across different data types.

Step 5: Model Training. Input the image dataset into Branch 1 for training. Simultaneously input the raw dataset into Branch 2 for training. Achieve fault classification through complementary feature extraction and feature fusion between both branches. Step 6: Model Testing. Evaluate the trained network using testing subsets. Obtain classification accuracy metrics for each fault type.

3.1. Experiment

To validate the effectiveness of the method proposed in this study, we used the Case Western Reserve University (CWRU) bearing dataset [43]. Experiments were then conducted and verified on the Jiangsu Normal University-Wind Turbine-1 (JSNU-WT-1) test bench. The specific details of the two experiments are as follows:

The CRWU datasets test bench is shown in Fig. 4. The test bench mainly consists of four parts: the motor, torque sensor/encoder, power meter, and an electronic controller. The bearing to be tested is connected to the motor shaft, with the drive-end bearing model being SKF6205 and the fan-end bearing model being SKF6203. This study focuses on the drive-end SKF6205 bearing, with a system sampling frequency of 12 kHz. The fault points on the bearing are created through electrical discharge machining, with fault damage diameters of 0.1778 mm, 0.3556 mm, and 0.5334 mm. There are three fault locations on the bearing: Inner Race Fault (IR), Rolling Element Fault (RE), and Outer Race Fault (OR). Therefore, the bearing fault states are categorized into 9 fault states and 1 normal state, making a total of 10 bearing states. The vibration data was collected using an accelerometer, which was fixed to the housing with a magnetic base. The accelerometer was placed at the 12 o’clock position on both the motor housing drive-end and the fan end. All data were collected under different motor load conditions (0, 1, 2, and 3 hp). This study utilizes drive-end data acquired under 0hp load conditions with sampling frequencies of 12 kHz and 48 kHz as experimental data.

Fig. 4CRWU test bench [43]

The design of the wind turbine fault simulation test bench is shown in Fig. 5 [1, 3]. The experimental data are obtained from the JSNU-WT-1 test bench, and the main research object of this experiment is the rolling bearings at the input end of the wind turbine gearbox. In order to simulate different types of bearing faults, these bearings are artificially damaged using wire electrical discharge machining (WEDM) technology to simulate various possible fault conditions. The entire testing process is precisely controlled by a control cabinet to ensure the stability of the experimental environment. During the experiment, the motor drives the planetary gearbox via a coupling, and the planetary gearbox is connected to the wind generator through a synchronous belt, thereby driving the generator to operate. The system’s rotational speed is regulated by a frequency converter, allowing for precise control of the speed. The signal acquisition system uses the INV3062T0 signal acquisition device, responsible for collecting various data from the entire system. for vibration signal acquisition, four INV9822 accelerometers are used, mounted on the gearbox housing and fixed with magnetic bases. The sampling frequency of the accelerometers is set to 4 kHz to ensure accurate capture of high-frequency signals. The core hardware configuration of the test bench includes a 400 W-rated generator, a 750 W-rated three-phase asynchronous motor, and an XW90 planetary gearbox with a gear ratio of 3. Through the combination of real wind turbine generators and transmission systems, the test bench simulates the actual operating environment of the wind turbine, aiming to closely replicate the true operating conditions of the wind turbine in order to obtain more accurate and meaningful experimental data. This experiment configures seven distinct bearing fault types, encompassing faults in two different components (bearings and gears), specifically comprising three gear fault variants, three bearing fault variants, and one healthy gear condition. The precise fault locations are illustrated in Fig. 6 [1, 3].

Fig. 5JSNU-WT-1 test bench

Fig. 6Wind turbine bearing failure in different locations

3.2. Data processing

We perform sliding window sampling on the data collected as described above, which helps to improve the accuracy of both time-domain and frequency-domain analysis, reduce the boundary effects caused by signal segmentation, and enhance the ability to capture the details of the signal. Each signal segment comprises 1024 data points. for the CWRU dataset, the sliding window step size is 512, with an overlap of 512 between adjacent segments, as illustrated in Fig. 7. for the JSNU-WT-1 dataset, a reduced step size of 200 is adopted to increase sample quantity, resulting in an overlap of 824 between signals. CWRU dataset encompasses 16 distinct bearing health states. Each state contains 200 samples, yielding a total of 3,200 samples. These 16 bearing fault types have been systematically labeled; detailed specifications are provided in Table 2. JSNU-WT-1 dataset includes 7 bearing fault conditions. Each condition comprises 200 samples, resulting in 1,400 total samples. These 7 bearing and gear fault types have undergone labeling procedures; comprehensive information is documented in Table 3.

Fig. 7Example of raw data split into n samples

3.3. Experimental setup and parameter settings

To enhance the model’s capacity for integrating heterogeneous data, strengthen the network’s representational capability and diagnostic accuracy, while improving system robustness and reliability, this study converts raw 1D signals into 2D time-frequency representations using an improved TSMST method. These transformed images exhibit richer features and concentrated energy distribution, forming the input for Branch 1 with dimensions of 256×256. The original 1D data serves as input for Branch 2 with dimensions of 1×1024. The dual-path architecture fuses these distinct data modalities to deliver comprehensive and precise fault diagnosis capabilities. The algorithmic parameters are configured as follows: initial learning rate = 0.0005, optimizer = Adam, batch size = 256, MaxEpochs = 20, L2 regularization parameter = 0.01, with softmax as the classifier.

In this paper, the parameters of all models are uniform. The algorithm was implemented using MATLAB. The computer used has an i5-12500H CPU and Windows 11 operating system. Additionally, the dataset was split, with 80 % used as the training set and 20 % used as the test set.

3.4. Performance evaluation metrics

In this study, to evaluate the advantages of the proposed algorithm model, we use four metrics: accuracy, recall, precision, and F1-score. Accuracy is the proportion of correctly predicted samples to the total number of samples in a classification model. It reflects the overall performance of the model across all samples and is suitable for cases with balanced data distribution. Recall measures the proportion of actual positive samples that the model correctly predicts as positive. in other words, it indicates how many of the actual positive samples the model is able to identify. Precision measures the proportion of predicted positive samples that are actually positive. in other words, it indicates how many of the samples predicted as positive by the model are truly positive. F1-score is the harmonic mean of precision and recall, considering the balance between the two. The F1-score is especially useful in cases of class imbalance, as it provides a trade-off between precision and recall. By combining both precision and recall, the F1-score offers a more comprehensive evaluation of model performance, and it provides a more reliable assessment, especially when the distribution of positive and negative classes is uneven. The definitions of these four metrics are as follows:

where $TP$ (True Positive) represents that a positive sample is correctly predicted to be positive, $TN$ (True Negative) represents that a negative sample is correctly predicted to be negative, $FP$ (False Positive) represents a Negative sample being incorrectly predicted as a positive, and $FN$ (False Negative) represents a positive sample being incorrectly predicted as a negative.

4.1. Case 1: CRWU dataset fault diagnosis

4.1.1. The impact of different learning rates on model performance

Generally, Learning Rate (LR) significantly influence model performance as they directly control the step size for parameter updates during training. Excessively large LR prevent loss function convergence and may cause oscillation near optimal values. Conversely, excessively small rates lead to slow convergence, requiring more training epochs. Thus, selecting an appropriate LR is critical. This study tested four learning rates (0.0001, 0.0005, 0.001, and 0.005) for model training and evaluation. Fig. 8 illustrates classification performance metrics across these learning rates, including mean accuracy, standard deviation (std), and maximum/minimum values from all trials. Results indicate that the proposed model achieved peak average accuracy (99.78 %) with LR = 0.0005, accompanied by the lowest std (0.08 %), demonstrating optimal stability. Although LR = 0.001 yielded perfect classification (100 %) in one trial, its mean accuracy (99.75 %) and std (0.18 %) were inferior to the LR = 0.0005 configuration.

Fig. 8Performance comparison of the model under different learning rates

4.1.2. Comparison with different data types

To highlight the advantages of the improved method, we transformed the one-dimensional raw signal into time-frequency diagrams using both the improved TFMST and the original TFMST methods, resulting in two fault samples. These samples, along with the raw signal, were used for training and testing the proposed model. Fig. 9 shows the time-frequency diagram of different fault categories obtained using the improved TFMST method. Fig. 10 shows the time-frequency diagram obtained using the original TFMST method. Fig. 9 and 10 represent different representations of the same signal. From the figures, it can be observed that the time-frequency diagram obtained using the improved TFMST method contains more features, which will benefit the diagnostic accuracy of the model.

Fig. 11 shows the average confusion matrix of the time-frequency diagrams obtained using two different methods tested on the proposed model. From the figure, it can be observed that the improved TFMST data only has the possibility of misclassification between fault categories 2 and 3 in the model. in contrast, the original TFMST data shows the possibility of misclassification for four fault categories, including category 1, category 2, category 3, and category 15. Additionally, there is confusion between fault categories 2 and 3. This indicates that the time-frequency diagram obtained using the improved TFMST method contains more features, resulting in better diagnostic accuracy in the model.

Fig. 9Improved TFMST Time-Frequency diagrams for different fault types

4.1.3. Comparison with different models

The DSC-CNN-GRU model incorporates two distinct branches: Branch 1 constitutes a lightweight pathway employing DSConv, while Branch 2 integrates CNN and GRU architectures with a similarly lightweight design approach. Consequently, the proposed model can be modularized into two sub-models denoted as Light-CNN and CNN-GRU. The LeNet-5 and AlexNet models represent classical CNN architectural paradigms characterized by differing computational complexities and historical significance, serving as benchmark references for foundational deep learning frameworks.

Fig. 10Original TFMST time-frequency diagrams for different fault types

Fig. 11Average confusion matrix of test results from different methods on the proposed model

a) Average confusion matrix results of the improved TFMST method

b) Average confusion matrix results of the original TFMST method

To demonstrate the superiority of the proposed model, these four architectures were established as comparative baselines, creating a comprehensive evaluation spectrum spanning elementary to advanced implementations and lightweight to relatively heavyweight configurations. This systematic comparison highlights the proposed model’s exceptional advantages in balancing performance and computational efficiency.

To highlight the advantages of the proposed model, we analyzed the results of multiple tests on the model using different metrics, including accuracy, precision, recall, and F1-score, as shown in Table 4. To provide a clearer comparison of the models' performance, we visualized these results in Fig. 12. The CNN-GRU and AlexNet models exhibited large standard deviation (std) fluctuations, indicating poorer model stability. Additionally, the average diagnostic accuracy of these two models was lower than that of the other models. in comparison to the LeNet-5 model, the Light-CNN model demonstrated higher stability and diagnostic accuracy. Notably, the proposed model achieved the highest diagnostic accuracy and best overall performance, with an accuracy of 99.78 %.

Table 4Performance comparison of different models

Model	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)
LeNet-5	99.22±0.2923	99.25±0.2533	99.22±0.2615	99.22±0.2533
Light-CNN	99.38±0.1914	99.40±0.1693	99.40±0.1693	99.38±0.1720
CNN-GRU	98.25±1.0678	98.58±0.7086	98.25±0.9550	98.27±0.9351
AlexNet	98.78±0.9968	98.86±0.7646	98.78±0.8916	98.77±0.8915
The proposed	99.78±0.0856	99.79±0.0747	99.78±0.0765	99.78±0.0865

Fig. 12Visualization comparison of the performance of different models

To further illustrate the diagnostic performance of different models on various fault categories, we plotted the average confusion matrix of the test results for each model, as shown in Fig. 13. The LeNet-5 model shows a potential for misprediction in five fault categories: categories 1-3, category 8, and category 15. The Light-CNN model has the possibility of misprediction in categories 1-4, but its accuracy is higher than that of the LeNet-5 model. The CNN-GRU model carries a risk of misprediction for categories 2-3, category 5, and category 15. The AlexNet model has the potential for misprediction across all six fault categories, resulting in the lowest prediction accuracy. in contrast, the proposed model only has a slight misclassification risk for faults in categories 2 and 3. Therefore, the proposed model demonstrates better performance, which is consistent with the results shown in Fig. 12.

Fig. 13Average confusion matrix of test results from different models

a) Lenet-5

b) Light-CNN

c) CNN-GRU

d) AlexNet

e) The proposed method

4.1.4. The effect of additional noise on model performance

Due to the complex operating environment of wind turbines, which is influenced by various environmental factors, the collected data can be affected by noise. in some harsh conditions, the impact of noise on the signal can be extremely severe. We added Gaussian noise with different signal-to-noise ratios (SNR) to the collected raw data to verify the classification performance of the proposed model under significant noise interference. the SNR can be defined as:

20

$SNR=10{\log }_{10}\left(\frac{P_{signal}}{P_{noise}}\right),$

where $P_{signal}$ and $P_{noise}$ are the original signal power and noise power, respectively. A smaller SNR value indicates stronger added noise and greater interference with the original signal. This presents a challenge for fault diagnosis models.

The proposed model, as a dual-input model, increases the input dimension of the network, allowing the model to learn potential patterns and relationships in the data from different perspectives. This structure enhances the model's expressive power, especially when dealing with complex input data, providing stronger generalization ability compared to single-input models. We added noise with different SNRs to the raw data and generated two image datasets using TFMST and the improved TFMST methods. Then, we performed comparative experiments on the proposed model using both datasets. The results are shown in Table 5, and we visualized these results. Fig. 14 displays the comparison of the two methods on accuracy, precision, recall, and F1-score under different SNR conditions. It can be observed that, under all conditions, the proposed model consistently demonstrated higher performance. Fig. 15 presents the average confusion matrix of the proposed model’s test results under the conditions of SNR = 0 dB and 15 dB with the original data. From the figure, it is evident that, although the proposed method predicts more fault categories incorrectly than the original method under different conditions, its overall prediction accuracy remains higher than that of the original method. Therefore, the proposed model demonstrates superior diagnostic performance with the improved TFMST data.

Fig. 14Visualization of comparative performance of different data on the proposed model under varying SNRs

Fig. 15Average confusion matrix of the proposed model’s test results under different SNR conditions

a) TFMST SNR = 0 db

b) Improved TFMST SNR = 0 db

c) TFMST SNR = 15 db

d) Improved TFMST SNR = 15 db

Table 5Comparison of different signal-to-noise ratio data on the proposed model

SNR (dB)	Method	Accuracy (%)	Recall (%)	Precision (%)	F1-score (%)
0	Improved TFMST	97.41±0.6592	97.43±0.5938	97.41±0.5896	97.40±0.5938
	TFMST	97.28±0.4222	97.28±0.3748	97.28±0.3776	97.24±0.3818
5	Improved TFMST	98.81±0.6011	98.86±0.5128	98.81±0.5376	98.80±0.5467
	TFMST	98.60±0.9440	98.73±0.6579	98.60±0.8443	98.60±0.8491
10	Improved TFMST	99.44±0.5015	99.44±0.4430	99.44±0.4485	99.44±0.4486
	TFMST	98.97±0.2370	99.01±0.2020	98.97±0.2119	98.97±0.2126
15	Improved TFMST	99.50±0.2318	99.51±0.1979	99.50±0.2073	99.50±0.2078
	TFMST	99.25±0.3563	99.28±0.2849	99.25±0.3187	99.25±0.3168

4.2. Case 2: JSNU-WT-1 fault diagnosis

4.2.1. Comparison with different data types

Fig. 16 and Fig. 17 show the time-frequency diagrams of the same signal for seven different fault types, obtained using the TFMST and the improved TFMST methods, respectively. From the figures, it can be observed that the time-frequency diagram obtained using the improved TFMST method contains more fault information. in general, the diagrams with rich information contribute to the correct predictions of the model. We input the two different datasets into the proposed model, and the average confusion matrix of the test results from both models is shown in Fig. 18. It can be seen that the proposed model demonstrated good diagnostic performance on the improved TFMST data. for the improved TFMST data, the proposed model only showed the possibility of misclassification for the ball fault and outer ring fault categories. for the original TFMST data, the model showed the possibility of misclassification for all fault categories. The proposed model exhibited excellent performance on the improved TFMST data.

Fig. 16Improved TFMST Time-Frequency diagrams for different fault types

Fig. 17Original TFMST time-frequency diagrams for different fault types

Fig. 18Average confusion matrix of test results from different methods on the proposed model

a) Average confusion matrix results of the improved TFMST method

b) Average confusion matrix results of the original TFMST method

4.2.2. Comparison with different models

To highlight the advantages of the proposed model, we compared it with LeNet-5, Light-CNN, CNN-GRU, and AlexNet. for the five different models, we compared the four metrics of accuracy, precision, recall, and F1-score, as shown in Table 7, and visualized the performance comparison of each model in Fig. 19. It can be seen that the metrics of the LeNet-5, Light-CNN, CNN-GRU, and AlexNet models are all lower than those of the proposed model, with LeNet-5, Light-CNN, and AlexNet models showing poor stability. Although the Light-CNN model exhibits good stability, its diagnostic accuracy remains lower than that of the proposed model.

Fig. 19Visualization comparison of the performance of different models

Fig. 20Average confusion matrix of test results from different models

a) Lenet-5

b) Light-CNN

c) CNN-GRU

d) AlexNet

e) The proposed method

Fig. 20 shows the average confusion matrix of the test results from different models. The LeNet model correctly classified only the faults of the 2nd, 3rd, and 6th categories, while it had a risk of incorrect predictions for the remaining four fault categories, with the highest likelihood of error occurring for the 4th category. Similarly, AlexNet model only correctly predicted the faults of the 3rd and 5th categories, while errors occurred in the predictions of the remaining faults, making its performance the worst. Although Light-CNN and CNN-GRU demonstrated relatively high diagnostic performance, the proposed model outperformed them, delivering the best overall results and performance.

Table 7Performance comparison of different models

Model	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)
LeNet-5	95.79±3.1984	96.67±1.9144	95.79±2.8607	95.63±1.9144
Light-CNN	97.50±0.4374	97.80±0.2896	97.50±0.3912	97.50±0.3756
CNN-GRU	92±2.3905	94.09±1.6417	92±2.1381	91.65±2.2714
AlexNet	94.00±3.5839	94.46±2.6605	94.00±3.2055	93.91±3.2746
The proposed	99.57±0.4657	99.59±0.3990	99.57±0.4165	99.57±0.4196

4.2.3. The effect of additional noise on model performance

To further validate the superiority of the proposed method, we added noise with different SNR values to the raw signal and obtained two different datasets using the improved TFMST method and the original TFMST method. We then conducted comparative experiments with these two datasets on the proposed model. The results were statistically analyzed using the four metrics: accuracy, precision, recall, and F1-score, as shown in Table 8. Additionally, to provide a clearer comparison, we visualized the experimental results, as shown in Fig. 21. From the results, it can be observed that under all conditions, the improved TFMST data exhibited higher performance on the proposed model and demonstrated greater stability.

Fig. 21Visualization of comparative performance of different data on the proposed model under varying SNRs

Table 8Comparison of different signal-to-noise ratio data on the proposed model

SNR (dB)	Method	Accuracy (%)	Recall (%)	Precision (%)	F1-score (%)
0	Improved TFMST	64.71$\pm$4.5904	67.94$\pm$4.0644	64.71$\pm$4.0157	65.00$\pm$4.0644
	TFMST	64.43$\pm$3.7593	65.96$\pm$2.9316	64.43$\pm 3.3625$	64.53$\pm$3.1596
5	Improved TFMST	88.07$\pm$4.8682	88.89$\pm$3.8397	88.07$\pm$4.3542	88.07$\pm$4.3809
	TFMST	87.29$\pm$3.4032	88.92$\pm$2.5271	87.29$\pm$3.0439	87.43$\pm$2.9494
10	Improved TFMST	97.29$\pm$1.3505	97.35$\pm$1.1723	97.29$\pm$1.2080	97.28$\pm$1.2066
	TFMST	92.86$\pm$1.4940	93.55$\pm$1.0898	92.86$\pm$1.3363	92.88$\pm$1.3114
15	Improved TFMST	97.93$\pm$0.6870	97.99$\pm$0.6188	97.93$\pm$0.6145	97.92$\pm$0.6269
	TFMST	94.50$\pm$2.3090	95.09$\pm$1.6802	94.50$\pm$2.0653	94.41$\pm$2.2183