Multilingual Voice and Vision AI Doctor

Abstract

Healthcare accessibility remains a major challenge in many rural and remote regions due to limited medical facilities, language barriers, and the shortage of healthcare professionals. To address these challenges, this project presents a Multilingual Voice and Vision AI Doctor, an intelligent healthcare assistance system that combines Artificial Intelligence, Speech Recognition, Medical Image Analysis, Natural Language Processing, and Text-to-Speech technologies. The proposed system enables users to describe their symptoms through voice input and upload medical images such as skin infections, wounds, and rashes for analysis. Speech input is converted into text using Speech Recognition, while medical images are processed using the Google Gemini 2.5 Flash Lite multimodal AI model. The system simultaneously analyzes both textual and visual information to generate preliminary healthcare recommendations. The generated response is then converted into natural speech, allowing users to receive guidance in their preferred language, including English, Hindi, and Kannada. Developed using Python and Gradio, the platform provides an interactive and user-friendly interface. Experimental results demonstrate effective performance in voice recognition, image analysis, multilingual communication, and response generation, making the system a valuable tool for accessible and intelligent healthcare assistance.