Visual Food Image Recognition Using Deep Learning

Food image recognition is the foundational computer vision task for the rapidly advancing field of computational nutrition and Automated Dietary Assessment (ADA). Precise food classification systems are essential for intelligent health management, providing valuable support for individuals managing chronic conditions such as obesity and diabetes by enabling reliable tracking of high-calorie food consumption. The ultimate objective is the real-time, accurate estimation of a meal’s nutrient content, which necessitates robust, high-performance visual classification as a prerequisite.Food image recognition remains a challenging Fine-Grained Visual Classification (FGVC) task due to high inter class similarity and significant intra-class variation caused by differences in food preparation, presentation, and viewpoint. Additionally, a substantial domain gap exists between curated benchmark datasets, such as Food-101, and complex real-world food images containing multiple items and visual noise. Although Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have achieved strong performance on controlled datasets, CNNs often underutilize fine-grained discriminative features, while ViTs suffer from high computational complexity. To address these limitations, recent research has explored hybrid architectures and feature enhancement techniques that combine global context with local detail. Building on this direction, this thesis evaluates contemporary deep learning architectures on rigorous benchmarks, including CNFOOD-241, with particular focus on efficient models capable of strong generalization in real-world food recognition scenarios.To address the limitations identified in prior approaches, this research proposes a Swin-S Transformer-based architecture enhanced with a Token-level Squeeze-and-Excitation (SE) mechanism, enabling adaptive feature reweighting and improved global–local feature interaction. Through careful architectural design and optimized training strategies, the proposed model achieves a Top-1 classification accuracy of 83.55%, outperforming the current state-of-the-art model by 1.85% on the same dataset. These results demonstrate that enhanced hierarchical Transformers can surpass recent SSM-based approaches in FGVC tasks while maintaining stable convergence and strong generalization capability. Overall, this work delivers a robust, high-performance food image classification module suitable for complex downstream applications, including hierarchical food categorization and future integration with Vision Language Models (VLMs) for comprehensive automatic nutritional assessment.