to supplement instructions with BLV-specific tips and non-visual workarounds (like using sound, smell, or touch). Proactive Feedback
According to research published at UIST 2025 and arXiv , the system aims to:
Millions of people use online video platforms daily to master new skills, such as: vid2coach top
is an innovative AI-driven system designed to transform standard how-to videos into interactive, wearable assistants, primarily developed for the blind and low-vision (BLV) community.
Vid2Coach is an innovative assistive technology system designed to bridge the gap between standard instructional videos and the needs of blind and low-vision (BLV) individuals. Traditionally, learning from "how-to" videos—whether for cooking, exercise, or crafts—requires a heavy reliance on visual comparison. Vid2Coach transforms these static videos into interactive, camera-based task assistants that provide real-time guidance and feedback. Top Features of the Vid2Coach System However, for a BLV individual, standard video narration
For sighted learners, these videos provide intuitive spatial and visual guidance. However, for a BLV individual, standard video narration is rarely sufficient. Traditional videos contain long silences, unvoiced physical gestures, and implicit cues (e.g., "cook until golden brown") that lack audio description.
By grounding large multimodal models (LMMs) in local first-person video data, Vid2Coach avoids the "hallucination" errors common to generalized AI assistants. It ensures that learning a new skill from the internet remains a fully accessible, hands-free experience. for a BLV individual
Vid2Coach: How AI is Transforming Online How-To Videos into Smart Wearable Coaches
, it pulls non-visual tips from BLV-specific community resources—for example, suggesting the use of kitchen scissors instead of a knife for safety. Proactive Feedback