پیوندهای مفید
آمار
| تعداد نشریات | 13 |
| تعداد شمارهها | 238 |
| تعداد مقالات | 2,418 |
| تعداد مشاهده مقاله | 3,961,104 |
| تعداد دریافت فایل اصل مقاله | 2,873,186 |
Deep Learning Attention-based Framework for Integrating EEG and Image Information in Visual Content Recognition | ||
| Journal of Electrical and Computer Engineering Innovations (JECEI) | ||
| مقالات آماده انتشار، پذیرفته شده، انتشار آنلاین از تاریخ 24 خرداد 1405 | ||
| نوع مقاله: Original Research Paper | ||
| شناسه دیجیتال (DOI): 10.22061/jecei.2026.12557.890 | ||
| نویسندگان | ||
| Hamed Hakkak؛ Mohammad Mahdi Khalilzadeh* ؛ Mahdi Azarnoosh؛ Hamid Reza Kobravi | ||
| Department of Biomedical Engineering, Ma.C., Islamic Azad University, Mashhad, Iran. | ||
| تاریخ دریافت: 03 بهمن 1404، تاریخ بازنگری: 22 اردیبهشت 1405، تاریخ پذیرش: 05 خرداد 1405 | ||
| چکیده | ||
| Background and Objectives: While deep learning has significantly advanced visual content recognition, existing models primarily rely on image data alone, neglecting the rich cognitive context embedded in neural responses. This study aimed to develop and validate a novel framework that synergistically integrates electroencephalography (EEG) signals with visual features to achieve superior accuracy in multiclass image recognition. Methods: We designed a hierarchical attention-based deep learning architecture to fuse neural and visual information. EEG data recorded (the dataset newly developed by the authors) during visual stimulus presentation were preprocessed and analyzed using temporal models (RNN-CNN and LSTM) to extract neural features. Concurrently, visual features were extracted from the stimulus images using ResNet101 and DenseNet201 architectures. The proposed attention mechanism dynamically weighted and integrated these multimodal features, prioritizing the most salient information from each modality. Results: The proposed framework significantly outperformed conventional unimodal approaches. The hybrid RNN-CNN + ResNet101 model achieved a peak classification accuracy. A feature contribution analysis revealed that the optimal performance was attained through an integrated contribution of approximately 60% from image-derived features and 40% from EEG-derived features, demonstrating the critical complementary value of neural data. Conclusion: This study confirms that the structured, attention-based fusion of neurophysiological and visual data substantially enhances visual content recognition. The findings provide a robust and effective framework for advanced cognitive assessment applications and establish a new benchmark for multimodal integration in machine learning, highlighting the significant potential of EEG data to complement and improve computer vision tasks. | ||
| کلیدواژهها | ||
| EEG–image Fusion؛ Attention-based Deep Learning؛ Multi-class Visual Content Classification؛ Hierarchical Attention Mechanism؛ RNN-CNN | ||
|
آمار تعداد مشاهده مقاله: 1 |
||