Motor bearing fault early warning model for TCN integrating multi-attention mechanisms

3.1. Cross-attention mechanism

The core of the CA lies in enabling a query sequence to focus on another key-value sequence, capturing dependencies between input parameters to achieve information fusion. As shown in Fig. 1(b), the CA architecture takes time-series parameters $F_t$, electrical parameters $F_p$, and mechanical parameters $F_{xyz}$ as inputs.

To achieve this fusion, the input features are first projected into the three key components of the attention mechanism: the Query, Key, and Value matrices. Specifically, the temporal features are linearly transformed to generate the $Q$ matrix. Simultaneously, the electrical features and mechanical features are concatenated and then jointly subjected to two separate linear transformations to produce the $K$ and $V$ matrices. These transformations are performed by learnable weight matrices, allowing the model to adaptively determine the optimal representation for querying and for being queried.

Once the $Q$, $K$, and $V$ matrices are derived, the attention scores are computed according to Eq. (4). This process allows the temporal features as the query to selectively attend to the most salient information within the combined electrical and mechanical features as the key-value source. Through this mechanism, the model effectively fuses heterogeneous sensor data, enhancing its ability to learn from fault-related patterns that manifest across different data streams:

where, $Q$, $K$ and $V$ are the matrices derived from the input features $F_t$, $F_p$, and $F_{xyz}$ as described above. ⨂ denotes matrix multiplication. $d$ is a scaling factor based on the key dimension, and $S$ represents the Softmax function calculates the final attention weights.

3.2. Temporal convolutional neural network with the squeeze-and-excitation mechanism

TCN can handle the strong temporal nature of motor operation data, addressing sequential issues during prediction while modeling dependencies in long-term [30] data within shorter timeframes. When combined with channel attention mechanisms, TCNs optimize weight distribution among data points, enabling greater focus on parameters with stronger simultaneous influence on faults. Their model structure is illustrated in Fig. 2.

The expansion causal convolutions and residual connections within the TCN effectively prevent gradient vanishing and overfitting. This ensures that the output at time t depends solely on the input data transmitted up to that point, avoiding confusion between historical and future information. By appropriately selecting the dilation factor and convolution kernel size, the model effectively processes long-term sequence information, expands the receptive field of network layers, and balances network depth and complexity, as detailed in Eq. (5):

where, $f_i$ denotes the filter function, $X_{t-d_i}$ represents the input data value shifted backward by time $d_i$ steps from time $t$, and ∗ indicates the convolution operation.

Residual connections address the issue of gradient vanishing caused by the continuous stacking of network layers due to the enlarged receptive fields from dilated convolutions. They also enable cross-layer transmission of multi-dimensional data information, enhancing computational efficiency. Within the residual block, the sequence of operations is arranged as dilated causal convolution, channel attention, weight normalization, activation function, and random deactivation, as shown in Fig. 2. Incorporating the channel attention mechanism before weight normalization enhances inter-channel information exchange, adaptively adjusts weights across different channels, and simultaneously improves the model’s generalization capability. The residual block is defined by Eq. (6), and the SE is defined by Eq. (7):

where, $\delta$ denotes the ReLU activation function, $G$ represents the five-part transformation comprising dilated convolutions, channel attention, weight normalization, activation functions, and random deactivation. $X_h$ and $X_{h-1}$ represent the outputs of the $h$th and $h-1$th residual blocks, respectively. $X_c$ and ${\widehat{X}}_c$ represent the channel features before and after SE weighting, respectively. $z$ represents the global features for each channel. $W_1$, $W_2$ are the weight matrices for dimensionality reduction and dimensionality augmentation operations. $\sigma$ denotes the sigmoid activation function.

Fig. 2Temporal convolutional neural with the squeeze-and-excitation architecture

3.3. Temporal pattern attention mechanism

To enhance the stability of temporal modeling and amplify the features of incipient faults, a the TPA is introduced. As depicted in Fig. 1(d), a TPA layer is strategically inserted between each pair of consecutive residual blocks within the Temporal Convolutional Network architecture. This placement allows the model to iteratively refine its focus on critical temporal patterns as data flows through the network.

The function of each TPA layer is to re-weight the feature sequence it receives. Specifically, the input to a TPA layer is the complete feature sequence output by the preceding TCN residual block. The TPA layer then internally uses a set of one-dimensional convolutional filters to assess the importance of different temporal patterns across the entire sequence, generating an attention score for each time step.

These scores are normalized and used as weights to compute a weighted sum of the original input sequence, a process summarized by Eq. (8). The output of the TPA layer is this re-calibrated feature sequence, which has the same dimensions as the input. This output then serves directly as the input for the subsequent TCN residual block:

where, $h_t$ represents the hidden states from the input sequence, $\sigma$ denotes the sigmoid activation function, $F_s$ represents the scoring function, $H_i$ denotes $\left\{H_{\left(i,j\right)}\right\}$ with the convolution kernel $j$ omitted, $T$ denotes matrix transposition, and $W_a$ denotes the learnable weight matrix. The resulting vector $V_t$ is the attention-optimized output sequence.

This iterative process of feature extraction by a TCN block and temporal re-weighting by a TPA layer) strengthens the model’s ability to capture subtle fault indicators over long sequences. The feature sequence from the final TPA layer is then passed to a fully connected layer to produce the ultimate motor bearing fault prediction $Y$.

4.1. Temporal fluctuation characteristics of active power under bearing failure

Active power serves as the core parameter reflecting a motor’s energy consumption and load status. When bearings are functioning normally, the rotor’s operational resistance remains stable. Under conditions where stator current, voltage, and power parameters are essentially constant, power generally maintains a steady range. The instantaneous active power $P_{inst}\left(t\right)$, as derived from Eq. (9), corresponds to the average active power $P_{avg}$:

After power is supplied to the motor stator, electrical energy is first converted into electromagnetic power $P_e$. However, this does not translate into mechanical power $P_m$ without losses, as stator-side losses must be considered. These primarily consist of stator copper losses $P_{cul}$ and stator iron losses $P_{fe}$. For simplified analysis, stator iron losses can be temporarily neglected due to their small magnitude. Therefore, the stator power can be approximated as $P_{avg}\left(t\right)\approx P_{inst}\left(t\right)=P_{cul}+P_e$.

When bearing failure occurs, the electromagnetic torque $T_e$ becomes less than the load torque $T_f$, causing the rotational speed n to inevitably decrease. Calculating the slip rate yields $s=(n_0-n)/n_0$. With the synchronous speed $n_0$ remaining constant, this decrease in rotational speed leads to an increase in electromagnetic power, thereby driving up the average active power.

For low-frequency time-series data, it is necessary to characterize the power fluctuation patterns within the sliding window. Define the sliding window length as $N$ and the sliding step size as $k=$1. For any time point $t_k$ corresponding to the sliding window, the active power data sequence it contains can be represented as $\left\{P\left(t_i\right)\right\}=\left\{P\left(t_1\right),P\left(t_2\right),\cdots ,P\left(t_{i+N-1}\right)\right\}$.

The time span aligns with the periodic characteristics of low-frequency signals, enabling real-time capture of trend changes in low-frequency active power timing. The step size slides sequentially by data points to ensure no omissions, thereby quantifying fault features within the window. This can be expressed using Eqs. (10-11) [31]:

where, ${\sigma }_P\left(t_k\right)$ denotes the power variance within the window, and ${\mu }_P\left(t_k\right)$ denotes the power mean within the window. $C_p\left(t\right)=\sqrt{{\sigma }_P\left(t\right)}/{\mu }_P\left(t\right)$ is the coefficient of time-series fluctuation for active power. Used to distinguish between “normal fluctuations” and “fault fluctuations” in low-frequency time-series data, it suppresses interference from steady-state power amplitude variations under different operating conditions, reflecting the steady-state power level within the observation window.

4.2. Temporal statistical characteristics of three-axis displacement vibration under bearing failure

The $X$, $Y$, and $Z$-axis vibration displacements are direct physical quantities reflecting the mechanical condition of the bearing. Their low-frequency time-series data encapsulate the macroscopic characteristics of rotor motion, better revealing the gradual progression of fault evolution compared to high-frequency signals. During normal operation, the $X$ and $Y$-axis radial vibrations maintain low-amplitude fluctuations due to the symmetrical rotation of the rotor, while the $Z$-axis axial vibration amplitude approaches zero. The time-series mean and standard deviation of all three axes remain within specified limits.

For low-frequency vibration data, the window design is consistent with that used in active power analysis. The data sequences it encompasses can be represented as $\left\{X\left(t_i\right)\right\}=\left\{X\left(t_1\right),X\left(t_2\right),\cdots ,X\left(t_{i+N-1}\right)\right\},\left\{Y\left(t_i\right)\right\},\left\{Z\left(t_i\right)\right\}$. Taking $X$-axis vibration as an example, the coefficient of variation ${CV}_x\left(t\right)=\sqrt{{\sigma }_x\left(t\right)}/{\mu }_x\left(t\right)$ is calculated to standardize the representation of vibration dispersion, thereby eliminating interference from variations in steady-state vibration amplitude across different operating conditions:

where, ${\sigma }_x\left(t_k\right)$ represents the standard deviation of the $X$-axis vibration window, indicating the intensity of data fluctuations within that window. ${\mu }_x\left(t_k\right)$ denotes the steady-state displacement baseline within the window.

4.3. Joint characterization of low-frequency fluctuations in bearing failure

Bearing failure fluctuations manifest as coupled variations in mechanical vibration and energy consumption, appearing in low-frequency time-series data as coordinated anomalies in active power and vibration displacement. During normal operation, stable mechanical resistance results in weak correlation between vibration and power fluctuations. Under fault conditions, sudden changes in mechanical impedance trigger strong coupled fluctuations. This state manifests as increased mechanical impedance disrupting torque balance, causing reduced rotational speed and increased slip rate. Consequently, average active power rises while three-axis vibration displacement exhibits abnormal fluctuations. Eq. (14) constructs characteristic vectors to comprehensively describe fault fluctuations:

Eq. (15) employs a sliding window Pearson correlation [32] to quantify the linear association strength between active power and $X$-axis vibration displacement within the same time window. Similarly, ${\gamma }_{Py}\left(t\right)$ and ${\gamma }_{Pz}\left(t\right)$ are obtained for the $Y$ and $Z$ axes, respectively, with correlation coefficients ranging between [–1, 1]. During normal operation, the mechanical system remains stable, and the Pearson correlation coefficient does not exhibit abnormal values near the boundary. Under fault conditions, periodic impacts cause highly synchronized trends in both variables, leading to a significant increase in the Pearson correlation coefficient. Moreover, abnormal peak values in a specific axis system can assist in locating the fault direction:

where, ${argmax}_{\tau }$ represents the maximum cross-correlation criterion [33], identifying the $\tau$ value that maximizes the summation term. Here, $\tau$ denotes the lag time, used to analyze the physical delay inherent in the power response transmitted through the rotor system during mechanical vibration impacts. Similarly, ${\tau }_{Py}\left(t\right)$ and ${\tau }_{Pz}\left(t\right)$ are obtained for the $Y$ and $Z$ axes. During fault conditions, mechanical impact energy is transmitted through the bearing-stator winding to trigger power changes. Therefore, the vibration impact must precede the power variation.

5.1. Bearing failure simulation method

Based on core requirements for industrial operation and fault monitoring, the system focuses on two typical states:

1) Normal operation state: Utilizes identical healthy bearings to confirm the absence of mechanical faults in the motor, while acquiring baseline electrical and mechanical parameters.

2) Bearing fault state: Emulates actual failure scenarios using artificially prepared faulty bearings to analyze abnormal parameter variations under different load conditions. The artificially induced bearing faults, as shown in Fig. 3(a), include insufficient lubrication, surface scratches on individual rolling elements, scratches and dents on the inner ring, and irregular damage to the rolling element cage.

To cover common industrial load scenarios, motor loads were set to no-load, 30 %, 50 %, 70 %, and 100 %. Under each operating condition, the motor first ran stably for 30 minutes before collecting data in both normal and fault states, ensuring consistent comparison benchmarks between the two states. As shown in Fig. 3(b), the primary equipment includes a three-phase squirrel-cage induction motor, gearbox, data acquisition unit, and magnetic powder brake. The data sampling frequency was set to 1 Hz. The magnetic powder brake adjusted the load and connected to the motor via sensors and the gearbox. Motor parameters are detailed in Table 1.

Fig. 3Bearing fault state & IM fabricated test rig description

a) Bearing fault state: 1 – lubrication deficiency; 2 – rolling element surface scratches; 3 – cage irregular damage; 4 – inner ring scratches and dents

b) IM fabricated test rig: 1 – three-phase squirrel-cage asynchronous motor; 2 – gearbox, data acquisition instrument; 3 – magnetic powder brake

Raw motor operating data is collected through the experimental platform. Under multiple operating conditions, non-steady-state data emerges during load gradient increases. The critical threshold for data stability is determined, and based on this, steady-state data outside this range is processed. The “3$\sigma$” criterion method is employed to detect and eliminate outliers. To prevent differing units and dimensions of various parameter characteristics from affecting fault prediction accuracy, preprocessed data undergoes MAX-MIN normalization. All parameters are mapped to the [0, 1] range eliminate the impact of dimensional differences between data features. The final dataset contains 10,000 bearing operational records. To ensure an unbiased evaluation, the dataset was partitioned into training, validation, and test sets using a 7:1.5:1.5 ratio through a stratified sampling methodology. The stratification was performed based on both the operating status (normal vs. fault) and the specific internal load condition of the fault data. This approach guarantees that the proportional representation of each category is preserved across all three sets, preventing evaluation bias and ensuring the model is tested on a representative distribution of the data, including rare fault conditions.

1) Operating Status: The overall ratio of normal to fault data is 74:26, which is maintained in the training, validation, and test sets.

2) Internal Motor Load Fault Data: Within the fault data, the distribution ratio for no-load, 30 %, 50 %, 70 %, and 100 % load conditions is 37:25:21:14:3, and this proportion is also consistently reflected in each dataset partition.

5.2. Model parameter configuration and evaluation criteria

The MA-TCN model was constructed within the PyTorch framework. Through debugging, the optimal iteration count was determined to be 100 with a batch size of 128. The bearing prediction model comprises a CA layer, TCN-SE layer, and TPA layer. Due to the integration of multiple attention mechanisms, network parameters are determined by the outputs of the preceding layers. Detailed parameter settings for each layer of the MA-TCN model are shown in Table 2.

The key hyperparameters presented in Table 2 were determined through a systematic process of empirical tuning, guided by the model’s predictive performance on a validation set. This iterative approach allowed for the refinement of the architecture to best suit the specific characteristics of the bearing fault data. For the dilation factors, the sequence [1, 2, 4, 8] was established as the most effective configuration. Preliminary evaluations with shallower dilation structures (e.g., [1, 2, 4]) resulted in a diminished receptive field and, consequently, a degraded ability to model long-term dependencies, leading to poorer performance. The kernel size was set to 3 after observing that it provided the optimal balance between local feature extraction and computational load. While smaller kernels failed to capture sufficient temporal context, larger ones did not yield a corresponding performance gain, thus making 3 the most efficient choice. Similarly, the channel dimensions were fine-tuned through experimentation. We observed that increasing the channel depth from 64 to 128 allowed the model to learn more complex and abstract feature representations, which significantly improved prediction accuracy. However, further increases in channel width led to diminishing returns and increased the risk of overfitting. Therefore, the final configuration in Table 2 represents an empirically optimized balance between model capacity and generalization performance for this specific application.

For the convolutional layers, the channel configuration [32, 64, 128, 256] follows a progressive expansion scheme to enable hierarchical feature extraction. The shallow layers capture low-level temporal patterns, while deeper layers learn more abstract representations. This design ensures sufficient expressive capacity with manageable computational cost. In the attention modules, the output dimensions of the CA, SE, and TPA layers were set to [16, 4, 128], [16, 256, 128], and [16, 256, 128], respectively, to maintain compatibility with TCN feature dimensions and preserve key channel and temporal dependencies. These values strike a balance between model expressiveness and training stability, as excessively large or small dimensions degraded performance during preliminary tuning.

To systematically validate the performance of the MA-TCN model in predicting low-dimensional unbalanced parameters for motor bearing faults, this study adopts a comprehensive evaluation framework incorporating six metrics: MSE, RMSE, MAE, MFE, MAPE, and $R^2$. Detailed calculation formulas for each metric are provided in Table 3.

Each tailored to address specific evaluative objectives for targeted model assessment:

– MSE: Sensitive to large errors, emphasizing significant deviations between predicted and actual values.

– RMSE: Reflects the actual error magnitude, providing an intuitive measure in the same unit as the original data.

– MAE: Captures the overall average error across all fault stages, enabling robust assessment of consistent performance throughout fault evolution.

– MFE: Identifies the model’s overall prediction bias, indicating tendencies toward overestimation or underestimation of fault parameters.

– MAPE: Compares relative accuracy across fault stages, supporting fair evaluation of relative deviations independent of the absolute magnitude of fault parameters.

– $R^2$: Measures goodness of fit to bearing fault evolution trends, assessing the model’s ability to capture the intrinsic pattern of fault development.

Notably, MSE, RMSE, MAE, MFE, and MAPE quantify prediction deviations via absolute or relative error metrics – lower values (closer to 0) denote higher local prediction accuracy. $R^2$ evaluates the model’s ability to align with actual data trends, with values approaching 1 indicating superior overall fit. While these error metrics are computed on the normalized [0, 1] data scale, they possess direct practical engineering significance. Since the model's output is a normalized representation of physical parameters, the metrics quantify prediction error as a fraction of the sensors' full operational range. In an industrial context, a low error value (e.g., a low MAE) signifies high fidelity between the model's prediction and the system’s true physical state. This is critical for engineering applications, as it ensures the reliability of the fault warning system and allows for the confident setting of precise alarm thresholds for timely preventative maintenance.

6.1. Derivative feature validation experiment and attention mechanism activation analysis

To evaluate the necessity and contribution of the derived features, we conducted 20 repeated training runs of the MA-TCN model. For each run, the Permutation Importance of both the original parameters $\left[P,X,Y,Z\right]$ and the ten derived features $[C_p,{CV}_x,{CV}_y,{CV}_z,{\gamma }_{Px},{\gamma }_{Py},{\gamma }_{Pz},{\tau }_{Px},{\tau }_{Py},{\tau }_{Pz}]$ was computed.

Fig. 4Derived feature contribution heatmap

Fig. 4 presents the Derived Feature Contribution Heatmap, which provides a visual representation of feature importance across the 20 repeated MA-TCN runs. As shown in the heatmap, the derived features consistently exhibit higher importance than the original parameters, confirming their substantial contribution to the model’s predictive performance. In particular, the coefficients of variation $[{CV}_x,{CV}_y,{CV}_z]$ demonstrate the most significant influence across nearly all runs, highlighting the effectiveness of capturing temporal-vibration fluctuation characteristics in supplementing the limited dimensionality of the original input data.

To validate the multi-attention architecture, we extracted activation patterns of its three core modules (CA, SE, TPA) from 30 test samples and computed three statistical metrics for each module. For each mechanism, three statistical metrics were computed: Mean (overall activation strength), Max (peak attention value), and Std (attention concentration degree).

Table 4 defines the nine attention indicators used in this analysis, detailing their definitions and physical interpretations within the MA-TCN.

Fig. 5 presents the Attention Mechanism Activation Heatmap, with each row representing a specific metric and each column corresponding to an individual test sample. The color intensity indicates normalized activation strength. Key observations include:

1) CA: exhibits variable ${CA}_{max}$ values across samples 2-8, indicating adaptive learning of inter-feature correlations critical to fault identification.

2) SE: maintains consistently high $S{E}_{mean}$ activation, confirming robust channel recalibration, while $S{E}_{std}$ variability in certain samples demonstrates the mechanism’s ability to differentiate channel importance based on fault characteristics.

3) TPA: shows pronounced ${TPA}_{max}$ peaks in specific samples 8-10, revealing successful identification of fault-critical time steps, which is essential for capturing transient anomalies in low-frequency data.

These complementary activation patterns confirm that the multi-attention architecture enables the MA-TCN model to effectively extract discriminative features from low-dimensional, imbalanced temporal data, validating the design rationale of integrating CA, SE, and TPA mechanisms.

Fig. 5Multi-attention mechanism contribution heatmap

6.2. Evaluation of ablation experiment results

The modified networks for the ablation experiments were TCN, NA-TCN (Non-Attention TCN), SE-TCN, CA-TCN, and TPA-TCN. These networks were trained under the same datasets, model configurations, and evaluation metrics as described above. The experimental evaluation results are shown in Figs. 6-7.

Fig. 6Evaluation results of different models in ablation experiments

Fig. 7Histogram of the frequency distribution of model prediction errors in ablation experiments

As shown by the experimental results in Fig. 6, the performance of CA-TCN, SE-TCN, and TPA-TCN significantly outperforms that of TCN and NA-TCN. This result indicates that integrating attention mechanisms with TCN effectively enhances the model's ability to extract valuable information from data while suppressing interference from irrelevant information.

Specifically, compared to TCN and NA-TCN, the R² values of CA-TCN, SE-TCN, and TPA-TCN all show notable improvements. Among these, CA-TCN introduces a the CA mechanism on top of TCN, learning dependencies among inputs in the data similarly to derived data; SE-TCN implements a channel attention mechanism to redistribute weights for critical fault information within residual connections; TPA-TCN leverages temporal attention to enhance the capture of fault time periods.

Based on the above analysis, this paper effectively integrates multiple attention mechanisms to propose MA-TCN. Ablation experiments demonstrate that compared to models combining a single attention mechanism with TCN, MA-TCN – which combines multiple attention mechanisms with TCN – exhibits superior predictive performance on special data. Experimental results show that MA-TCN achieves optimal metrics across all evaluation indicators, with an $R^2$ value as high as 0.9629. Furthermore, comparing the error distributions across models in Fig. 7 reveals that MA-TCN exhibits a Gaussian distribution of prediction error frequencies. Its prediction errors converge to a narrow range, indicating high prediction accuracy. These evaluation metrics collectively validate the high consistency between MA-TCN's predictions and actual values, demonstrating the model's strong fitting capability for predicting low-frequency, low-dimensional imbalanced data.

6.3. Evaluation of comparison experiment results

To ensure a fair and rigorous comparison, all baseline models were implemented based on their well-established, publicly recognized architectures, and subsequently tuned for the specific task of bearing fault diagnosis. Specifically, the architectures for LSTM and Transformer were adapted from their seminal works [34-35], while the ResNet model utilized a one-dimensional adaptation of the canonical residual learning framework [36]. The more recent TSMixer baseline was implemented according to its original proposed structure [37].

For each of these models, a consistent empirical tuning process was applied, focusing on optimizing key hyperparameters such as the number of layers, hidden unit dimensions, and learning rate to ensure each model achieved a robust performance level on our dataset. This approach guarantees that the comparison is not merely against default configurations, but against strong, representative implementations of each baseline, thus providing a fair benchmark for evaluating our proposed MA-TCN model.

Fig. 8Evaluation results of different models in comparison experiments

As observed in the prediction results of Fig. 8, similar to how LSTM leverages its gated recurrent feature to handle variable-length, non-uniformly sampled sequences, ResNet effectively mitigates the vanishing gradient problem in long sequence training through its fixed residual branch design. While LSTM achieves parameter sharing in the temporal dimension, ResNet accomplishes convolution kernel sharing in the spatial dimension. This difference results in comparable evaluation metrics for both models on the same dataset.

TSMixer, with its minimalist network architecture and lack of receptive field constraints, exhibits low overfitting risk and strong generalization stability. However, its fixed MLP module weights hinder dynamic parameter adjustment for derived data, resulting in slightly lower prediction performance than LSTM and ResNet. Additionally, while Transformers offer advantages in modeling global dependencies and parallel processing, they currently perform poorly in low-dimensional data scenarios.

Fig. 9Histogram of the frequency distribution of model prediction errors in comparison experiments

The MA-TCN model constructed in this paper effectively models dependency relationships under special data conditions compared to other networks. It derives feature data, dynamically adjusts fault period weighting in real-time based on physical rules, reduces the contribution of redundant channels, and accurately captures data failure timepoints. This enhances the model's feature learning capability and improves final prediction accuracy, achieving an $R^2$ value of 0.9629. Moreover, the error frequency distribution results in Fig. 9 indicate that the MA-TCN exhibits superior frequency distribution characteristics. This further validates the MA-TCN’s outstanding fitting performance in this prediction task.