Add support for local video and caption

[Ovler-Young]

As a student, I often record my classes and transcribe them locally so I can rewatch lectures and study more effectively. I would like to use Hyperaudio to view these recordings as interactive transcripts for better retention and understanding. However, loading local videos and captions currently seems difficult. They will be blocked by the browser and I found that fixing it is beyong my ability as an bioengineering student.

I transcribed recordings using the offline faster-whisper tool whisper-standalone-win as I can tune prompts for better results. It outputs subtitles in .srt format and a JSON with word timestamps. Samples are 2023-11-24 09_50_00_VGA.srt.zip and 2023-11-24 09_50_00_VGA.json

It would greatly improve the student workflow if Hyperaudio could support local video and captions, rather than requiring videos/transcripts to be hosted online. Would it be possible to add support for local video playback and caption loading? Supporting the whisper timestamp JSON format would also allow accurate syncing between video and captions!

[maboa]

Hi @Ovler-Young – I have a number of thoughts on this, I hope it's ok to convert to a discussion for further, er discussion :)