Auto adjust Subtitles based on Audio length

[telecentru]

Hello. I would like to request a new feature (or an explanation if this feature is already implemented without my knowledge).
So, I have an audio file (audio.mp3) and a plain text file with continuous text, without any timestamps (sub.txt) that is the text content of the audio file. When I import the text file into Subtitle Edit, it does generate timestamps, but depending on multiple factors, the duration of the subtitles is either shorter or longer than the audio.mp3. Usually the difference is small, but it needs manual adjustments.

The best feature in this case would be to be able to input the desired duration of the subtitle in Subtitle Edit (that is the duration of the audio.mp3) / or Subtitle Edit would automatically take the audio duration and provide with an option (checkbox) for the SRT to match the duration of the audio /, and the subtitle generation would take into account this duration and the correspondence between .srt and .mp3 would be maximum.

Thank you.

[miaochanglei]

good idea

[darnn]

Sure, it's a great idea, but who's going to implement it? It's not exactly simple. I suggest you look up voice activity detection (VAD) and forced alignment, which is what this is actually called. WhisperX seems to do it with something called wav2vec2, so maybe check that out. Either way, it's not trivial to just plop it into SE.

If it was me, and I was in this situation right now (an accurate transcript with no timecodes and an audio file), I would use WhisperX to produce an SRT file with more or less accurate timecodes. Then I'd import the accurate transcript into SE and save that as an SRT with no timecodes. Then I'd use SE's translation mode, open the WhisperX file and use the accurate transcript file as the "translation", to match the text to the timecodes manually. It's a hassle, but it's much quicker than doing it from scratch.

[miaochanglei]

thank you for you share, WhisperX refers to Whisper.cpp?

[darnn]

No, it refers to this:
https://github.com/m-bain/whisperX
I got it running on Windows using ESL. I tried the method outlined there using Conda but got a bunch of errors I wasn't able to resolve. If you do go the ESL route, note that I had to create a new ESL user, since installing the latest version of Python in the default user didn't work properly.