Multimodal Deep Learning for Diabetic Retinopathy Grading: Integrating Linear-Radon Sinograms and Retinal Fundus Images

Automated diabetic retinopathy (DR) grading is crucial for disease monitoring and personalized treatment, challenged by high intra-class variation and data imbalance. This paper presents a novel approach to enhancing DR grading detection by integrating linear-Radon sinogram-based images with original retinal images to provide a multimodal network using different convolutional neural network (CNN) architectures. Using the Kaggle Aptos dataset, we evaluated the performance of this multimodal integration. Our findings reveal a significant improvement in multi-class classification performance compared to unimodal retina-only images, underscoring our method's ability to detect subtle patterns among different DR grades. This study underscores the potential of sinogram-based images as a valuable modality for DR grading and paves the way for future research to validate this approach across diverse datasets and explore the application of curve-based sinograms.