Facial Emotion Recognition through Fine-Tuning of Pre-Trained Deep Learning Models

Facial Emotion Recognition (FER) in real-world, uncontrolled environments is a difficult task due to changes in pose, blockage, and lighting. This often forces a hard choice between the broad context understanding of Vision Transformers and the speed of Convolutional Neural Networks (CNNs). To solve this, this dissertation proposes ResEmoteNet-CBAM, a hybrid deep learning model that adds Convolutional Block Attention Modules (CBAM) to a residual network backbone to help the model better distinguish between features without slowing it down significantly. A detailed comparison was done on the Real-world Affective Faces Database (RAF-DB), testing the proposed model against three top existing models: the original ResEmoteNet, an improved VGG16 (VGG16-NFB), and the Vision Transformer APViT. Experiments show that ResEmoteNet-CBAM achieves a top-tier test accuracy of 97.43%, performing better than both the original ResEmoteNet (94.76%) and the APViT transformer (91.98%). Most importantly, adding spatial attention made the model much better at telling apart similar emotions, correctly identifying 90.14% of the difficult 'Fear' cases. Also, efficiency tests on an NVIDIA RTX 3090 showed the model is fast enough for real-time use, processing images in just 0.79ms (1269 FPS). The study concludes that hybrid models, which combine the structure of CNNs with lightweight attention tools, are a better choice than pure Transformers for medium-sized FER tasks, offering high accuracy without sacrificing speed.