Real-Time Pharmaceutical Drug Identification on Mobile Devices Using YOLO to Assist the Visually Impaired with Voice Assistants

Closed

Rezy Septianda, Kahlil Muchtar, Sayed Muchallil, Maya Fitria, Al Bahri, Novi Maulina

2025 Proceeding - 2025 International Conference on Computer System, Information Technology, and Electrical Engineering, COSITE 2025 Conference paper Cited by 0 Quartile

Abstract

Medication management poses a significant risk for visually impaired (VI) individuals, often leading to medication errors and a dependency on caregivers. To address this, technology-based solutions are crucial for enhancing safety and user autonomy. The proposed approach is a real-time medication identification system developed as a mobile application using the Flutter framework. At its core, the application utilizes a custom-trained You Only Look Once (YOLO) v11 object identification model, which runs entirely on-device to ensure privacy and functionality without an internet connection. The system identifies pharmaceutical packaging from a smartphone's live camera feed and delivers multimodal feedback through a clear voice assistant and haptic vibrations. The model was trained on a custom dataset of 800 images across six medication classes. Evaluation on the validation set demonstrated excellent performance, achieving a mAP@0.5 of 0.978 and a mAP@0.5:0.95 of 0.895. ©2025 IEEE.

Affiliations

Department of Electrical and Computer Engineering, Universitas Syiah Kuala, Aceh, Indonesia; Faculty of Medicine, Universitas Syiah Kuala, Aceh, Indonesia

Research at a Glance

Premium content — register to unlock

Research at a Glance

Register to unlock

Topics & SDG Alignment

Premium content — register to unlock

Topics & SDG Alignment

Register to unlock

Collaboration

Premium content — register to unlock

Collaboration

Register to unlock

Author Profile (Selected)

Premium content — register to unlock

Author Profile (Selected)

Register to unlock

References Overview

Premium content — register to unlock

References Overview

Register to unlock

Journal & Source

Premium content — register to unlock

Journal & Source

Register to unlock

Metadata & Integrity

Premium content — register to unlock

Metadata & Integrity

Register to unlock