DR-VQA: Large Language Model Based Vision Question Answering System on Diabetic Retinopathy

Medical Visual Question Answering (VQA) presents challenges due to the complexity of imaging data and the need for precise, context-aware responses. Traditional VQA models often struggle in clinical settings, limiting their utility in decision-making. This study proposes DR-VQA, a fine-tuned BLIP (Bootstrapped Language-Image Pretraining) model for medical VQA specifically designed for diabetic retinopathy (DR) based on fundus imaging, leveraging both visual and textual data to generate accurate diagnostic answers. To enhance semantic relevance, BERT-based similarity evaluation is integrated. Using a diabetic retinopathy dataset, the model achieves a validation BERT similarity (BERTsim) score of 0.94 and a test score of 0.95 on 450 samples, demonstrating strong alignment with expert annotations. These results highlight the model's potential to assist clinicians by improving diagnostic accuracy and efficiency. The proposed approach can streamline medical workflows, reduce clinician workload, and enhance patient outcomes. Future work will focus on expanding datasets and refining the model for broader medical applications. We believe our approach will support to enhance the patient care as well democratization on AI technology for community.