Research on multi-feature fusion fatigue driving detection algorithm based on improved YOLOv8

1.2.1. Physiological-signal-based fatigue detection

Physiological-signal-based methods identify fatigue by monitoring variations in electroencephalogram (EEG), electrocardiogram (ECG), electromyogram (EMG), and other physiological signals under fatigued conditions. Wang et al. evaluated EEG-based fatigue classification in a real driving environment using a wavelet scattering network [2]. Lees et al. investigated psychophysiological responses during monotonous driving and analyzed fatigue and sleepiness in train and non-professional drivers, providing driver-safety implications [3].

With the development of wearable sensing devices, increasing attention has been paid to combining physiological signals, such as electroencephalogram (EEG), electrooculogram (EOG), electrodermal activity (EDA), EMG, and ECG, with machine-learning algorithms. Shi and Wang investigated fatigue driving detection using time-space-frequency features from multimodal signals [4]. Li et al. developed a hybrid ensemble CNN framework for driver-fatigue recognition from EEG data [5]. Zhou et al. studied EEG fatigue-driving detection using functional brain-network features and machine-learning classifiers [6]. Gong et al. integrated pulse-wave and skin-conductance signals and established a fatigue-state database to achieve efficient fatigue detection [7].

Although physiological-signal-based methods can achieve high recognition accuracy, they usually require wearable sensors. This requirement increases user discomfort, introduces individual-dependence problems, and limits their feasibility for long-term in-vehicle deployment.

1.2.2. Vehicle-behavior-based fatigue detection

When a driver becomes fatigued, vehicle-control behaviors, including steering-wheel operation, grip, pedal input, and speed regulation, may change. Li et al. assessed fatigue from steering-wheel grip force [8]. Satti et al. developed an EMG-based system using steering-wheel grip for drowsiness detection [9]. Babusiak et al. proposed a smart steering wheel for unobtrusive health and drowsiness monitoring [10]. Houshmand et al. combined EEG signals with vehicle-dynamics data to identify moderate drowsiness [11]. Li et al. further used an LSTM network to extract fatigue-related features from steering-angle sequences, thereby improving detection accuracy [12].

Vehicle-behavior-based methods are non-intrusive and low cost. However, their performance is sensitive to road conditions, vehicle dynamics, and individual driving styles, which can reduce recognition stability under complex driving conditions.

1.2.3. Facial-feature-recognition-based fatigue detection

Facial behaviors associated with fatigue, such as blinking and yawning, provide intuitive visual cues and have become an active research direction. Bai et al. developed a two-stream spatial-temporal graph convolutional network that uses facial landmarks for driver-drowsiness detection [13]. Albadawi et al. used visual eye, mouth, and head-pose features for real-time driver-drowsiness detection [14]. Majeed et al. proposed a deep convolutional neural-network model that uses facial cues, including mouth behavior, for driver-drowsiness detection [15].

Deep-learning methods have recently been widely applied to this task. Florez et al. developed a real-time embedded system that combines convolutional neural networks with eye and mouth analysis [16]. Nomura et al. investigated the use of FaceMesh landmarks to reduce the effect of face orientation in drowsiness estimation [17]. Adhinata et al. integrated deep learning with traditional machine-learning methods to improve detection speed and robustness [18].

Facial-feature-recognition methods are non-contact and suitable for integration into intelligent driving systems. Nevertheless, their accuracy is easily affected by illumination variation, facial occlusion, camera viewpoint, and image quality. These challenges are especially prominent for small facial targets such as the eyes and mouth, where missed detections and false detections remain common. Consequently, fatigue-detection models require stronger small-target representation, robustness, and real-time inference capability.

1.2.4. Summary

In summary, current fatigue-driving detection technologies mainly follow three technical routes: physiological-signal analysis, vehicle-behavior analysis, and facial-feature recognition. Physiological-signal methods infer fatigue from EEG, ECG, EMG, and related signals. Although they can provide high recognition accuracy, their reliance on additional sensors may interfere with the driver and restrict practical deployment. Vehicle-behavior methods infer fatigue from steering-wheel angle, vehicle speed, pedal operation, lane deviation, and other dynamic indicators. These methods are non-contact and inexpensive, but their stability and generalization are limited by road conditions, vehicle type, and driving style. In contrast, facial-feature-recognition methods have attracted increasing attention because they are non-intrusive, efficient, and suitable for real-time monitoring. However, their practical performance is still constrained by lighting changes, occlusion, head-pose variation, and the difficulty of detecting small targets such as the eyes and mouth. A comparison of the principal characteristics of these three approaches is provided in Table 1.

The YOLO (You Only Look Once) series provides a lightweight one-stage detection framework with high inference speed and competitive accuracy [19]. Recent research has shown that YOLOv8 provides a practical baseline for image- and video-based object detection when trained on task-specific data [20]. Contemporary work has further developed scalable multi-scale real-time detectors, while recent reviews document the evolution and application of YOLO-based models [21, 22].

On this basis, this paper proposes FWC-YOLOv8 for driver fatigue detection. The method is not a simple stacking of existing modules; rather, it redesigns the detection scale, attention-enhanced feature extraction, multi-scale feature fusion, and bounding-box regression loss around the small facial targets that are most relevant to fatigue. These improvements are then coupled with a weighted multi-feature fatigue-assessment strategy to convert detected eye and mouth states into interpretable fatigue levels.

The main contributions of this work are as follows: (1) a four-scale detection structure is designed for fatigue-related small targets by adding a 160×160 detection head; (2) CA is introduced into the backbone to improve position-aware representation of the eyes and mouth; (3) WIoU is adopted to reduce the influence of low-quality bounding boxes and improve regression quality; and (4) a weighted multi-feature fatigue-assessment strategy is developed to fuse PERCLOS, blink frequency, continuous eye-closure duration, and yawn frequency for interpretable fatigue-level classification.

Specifically, the proposed feature-fusion optimization introduces an additional 160×160 detection branch into the multi-scale pathway, in which high-resolution spatial cues are integrated with semantically enriched features obtained through upsampling prior to small-target prediction. By preserving fine-grained spatial information while enhancing semantic discrimination, this modification improves the representation of small facial targets.

2.3.1. Model structure optimization

In fatigue-driving detection, the eyes and mouth occupy a small proportion of the image, so the network must retain sufficient spatial detail to locate and classify these regions accurately. Adding a 160×160 detection head allows the model to preserve higher-resolution features and improves its ability to detect small targets. Compared with the three-head structure of the original YOLOv8s, the four-head structure provides broader multi-scale coverage. A comparative experiment was conducted using the same training and validation protocol as the YOLOv8s baseline. As shown in Table 2, the added small-object detection layer increases mAP@0.50 by 2.1 % and mAP@0.50:0.95 by 2.4 %. Although GFLOPs increase, the number of parameters decreases slightly, indicating that the added head improves small-target accuracy while maintaining an acceptable accuracy-complexity trade-off.

2.3.2. Coordinate attention module

Attention mechanisms are widely used to improve model accuracy because they help the network emphasize informative regions and suppress irrelevant background responses. Recent comparative work has examined SE and CBAM as channel-oriented and joint channel-spatial attention mechanisms, respectively [25]. Attention-based feature learning has also been applied to driver-fatigue detection [26]. Although these attention mechanisms can improve feature extraction, they do not necessarily preserve spatial coordinate information explicitly. In fatigue detection, subtle movements of the eyes and mouth, such as blinking and yawning, are critical for determining fatigue level. Therefore, target position, shape, and orientation contain important information. This study introduces CA to address the limited spatial-coordinate modeling of conventional attention mechanisms in fatigue-driving detection.

CA explicitly encodes coordinate information through horizontal and vertical pooling operations on the feature map. This design helps the model learn the shape and orientation patterns of the eyes and mouth more effectively. By capturing long-range dependencies while preserving positional information, CA enhances the spatial representation capability of the network and improves the detection of small fatigue-related facial targets.

Fig. 3Comparison of the SE, CBAM, and CA attention mechanisms

In Fig. 3 show the SE, CBAM, and CA structures, respectively.

First, the feature map $F$ in $R^{\left(C\times H\times W\right)}$ is used as the input, where $C$ denotes the number of channels and $H$ and $W$ denote the height and width of the feature map, respectively. Coordinate information is encoded through horizontal and vertical average-pooling operations.

Horizontal pooling:

Vertical Pooling:

where $F_h$ and $F_w$ denote the feature maps encoded in the horizontal and vertical directions, respectively.

The encoded coordinate information is then recalibrated through two parallel 1×1 convolution layers to learn channel dependencies.

Horizontal feature re-calibration:

Vertical feature re-calibration:

$\sigma$ where the corresponding term denotes the activation function.

The recalibrated horizontal and vertical features are then fused through concatenation and a softmax operation.

Feature merging:

Softmax operation:

$F_{att}$ where the generated terms denote the attention weights.

Finally, the attention weights are multiplied by the input feature map to obtain the CA-enhanced output feature map $F_{out}$:

The CA mechanism integrates spatial coordinate information and improves the spatial representation of feature maps. This advantage is particularly useful for small facial regions such as the eyes and mouth, where subtle changes in eye opening and mouth shape determine fatigue cues. After adding CA, the model size increases only slightly, while detection accuracy improves by 2.1 %.

2.3.3. Improvement of loss function

The loss function guides learning and optimization in object-detection models. In YOLOv8s, classification loss is calculated using binary cross-entropy (BCE), while bounding-box regression is optimized using Distribution Focal Loss (DFL) and CIoU loss. CIoU evaluates the overlap between predicted and ground-truth boxes while considering aspect ratio and center-point distance. However, CIoU can be less effective under large scale variation, rotated target appearance, and abnormal low-quality labels. These limitations may reduce regression quality in complex driving scenes. Therefore, this paper introduces Wise-IoU (WIoU) as the bounding-box regression loss.

First, the IoU between the ground-truth box and the predicted box is calculated as follows:

where $A$ and $B$ denote the predicted and ground-truth bounding boxes, respectively. The intersection area is their overlap, and the union area is the total area covered by both boxes.

The distance between the center of the ground-truth box and the center of the predicted box is then calculated as follows:

where $\left(x,y\right)$ denotes the center of the predicted box, and $\left(x_{gt},y_{gt}\right)$ denotes the center of the ground-truth box. $W_g$ and $H_g$ are the width and height of the smallest enclosing rectangle containing both boxes.

The double-layer attention term $L_{W}IoU{v}_1$ is then calculated from the base IoU attention and the distance attention $R_{W}IoU$:

where the monotonic focusing coefficient is introduced to reduce the contribution of easily identifiable samples:

where $L_{W}Io{U}^{*}$ is the exponential moving average of the $IoU$ loss, and gamma controls the focusing strength.

Finally, a dynamic non-monotonic focusing hyperparameter $r$ is introduced to reduce the influence of abnormal low-quality bounding boxes:

where $r$ is dynamically adjusted according to the anomaly degree beta of the predicted box, as follows:

The anomaly degree beta is calculated as follows:

where $\alpha$ and $\delta$ are focusing-control hyperparameters and are usually set to 1.9 and 3.0, respectively.

WIoU is more complex than CIoU because it introduces additional weighting and focusing terms, and its hyperparameters require experimental adjustment. However, on the same dataset, WIoU avoids calculating aspect ratios for a large number of bounding boxes. The ablation results in Table 3 show an improvement in mAP@0.50:0.95 after CIoU is replaced with WIoU.

4.1.1. PERCLOS

PERCLOS is a widely used indicator for evaluating driving fatigue. It is defined as the proportion of time during which the eyes remain closed within a specified time interval. A recent evidence review describes PERCLOS as an established technology for drowsiness detection and notes its relevance to fatigue monitoring [27].

The proposed algorithm calculates PERCLOS as the ratio of closed-eye frames to total frames within a 25-s interval:

where $T$ denotes the unit time window of 25 s, and fps denotes the video frame rate.

4.1.2. Blinking frequency

Blinking is a normal physiological behavior, and the blink pattern of an alert driver generally remains relatively stable. Under fatigue, however, changes in ocular dryness and alertness may alter blinking behavior. Contemporary vision-based driver-monitoring systems therefore use closed-eye and blink events as fatigue-related cues [28]. In the state sequence ($N_k$), one complete blink is represented by a transition from ‘0’ to ‘1’ and then back to ‘0’. By counting the number of complete blink transitions within a given time interval, the blink frequency can be calculated as follows:

where Blink denotes blink frequency, and $n_{blink}$ denotes the number of blinks within the unit time $T$.

4.1.3. Duration of continuous eye closure

When a driver is fatigued, the duration of eye closure increases significantly. Under normal driving conditions, drivers instinctively avoid long eye-closure events for safety reasons; therefore, continuous eye closure indicates a high-risk driving state, regardless of whether other fatigue cues are present [29]. The algorithm defines $T_{blink}$ as the duration for which the eyes remain continuously closed:

where $n_s$ denotes the frame at which the eye state in queue $N_k$ changes from open to closed, and ne denotes the frame at which it changes from closed to open.

Studies show that alert drivers typically blink 20-30 times per minute, and each eye-closure event usually lasts 0.2-0.3 s. Based on these physiological characteristics, the fatigue-determination rules are defined in Table 4.