Emotion-Aware Contextual Modelling for Robust Driver Fatigue Detection

Vision-based driver fatigue detection remains challenging because facial signals associated with fatigue are often ambiguous, while geometric indicators such as Eye Aspect Ratio (EAR) and Percentage of Eye Closure (PERCLOS) are prone to false positives caused by normal facial activity, including smiling or speaking. This paper proposes a context-aware framework that integrates behavioural, geometric, and emotional information for robust fatigue assessment. Facial landmarks are extracted using MediaPipe Face Mesh, while adaptive eye-closure detection is performed through multi-stage validation combining EAR trajectories, mouth activity, head-pose analysis, and event-level filtering. Emotion recognition is achieved using an EfficientNet-B0 convolutional neural network trained on the AffectNet dataset, enabling frame-level estimation of facial expression probabilities. These predictions are aggregated into descriptors representing emotional variability and fatigue-related emotional relevance over time. Behavioural information obtained from blinking, yawning, head nodding, and validated PERCLOS is fused with emotional context to construct a multi-level fatigue assessment model. The final Driver Fatigue Risk Index combines physiological eye-closure information with contextual behavioural–emotional analysis, providing an interpretable estimation of driver state rather than a binary classification alone. Experimental evaluation on the NTHU-DDD dataset achieved 94% accuracy and demonstrated improved robustness under non-frontal head poses and expressive facial behaviour.