Trying to get hardcoded subs from video using InpaintDelogo

[eXtremeDevil]

What the title says. I'm not new to Avisynth and scripting but I'm also not an advanced user, so far I have this:

https://i.ibb.co/XxZ5VTP6/photo-2025-02-04-13-26-36.jpg (https://ibb.co/N26N7zqf)

I'm stuck here, the video is a YV12 already. Even if the video opens, I wouldn't know what to do next. I know I need to extract the subs with InpaintDelogo and the apply an OCR on it.

How can I fix this? Any better options to extract hardcoded subs?

Thanks in advance.

[StainlessS]

Maybe update your mvtools.
http://avisynth.nl/index.php/MVTools

External filters (On Wiki):- http://avisynth.nl/index.php/External_filters
Also, might be necessary to be using Avisynth+.

EDIT:

Avs+ Latest :: Pinterf on D9:- https://forum.doom9.org/showthread.php?t=181351
:: Github:- https://github.com/AviSynth/AviSynthPlus/releases
:: And Development Clang, IntelLLVM builds:- https://gitlab.com/uvz/AviSynthPlus-Builds

This one I think[ AviSynth+ 3.7.3 ::: 16th July 2023]:- https://github.com/AviSynth/AviSynthPlus/releases

EDIT: Sorry for the nonsense, you want MaskTools2 update not MvTools update (but check both updated a good idea).
Wiki:- http://avisynth.nl/index.php/MaskTools2
Pinterf/Masktools2 2.2.30 [19th Feb 2022] :- https://github.com/pinterf/masktools/releases/

[eXtremeDevil]

Maybe update your mvtools.

Yes, I'm using latest mvtools and Avisynth+...

[StainlessS]

Sorry, see previous EDIT:

EDIT:
From Masktools2 Wiki:- "supports RGB (planar) under Avisynth+",
that might be the reason. [I think is internally converted to RGB for subs detect]

[eXtremeDevil]

Sorry, see previous EDIT:


Nope, that one was updated as well... but, I tried with AvsP x64 instead of AvsP x86 and now is loading something, although is taking a lot of time... in case it finishes, what should I do next to extract the "sub" in a format I could then use with an OCR app, like SubtitleEdit an such?

[StainlessS]

From InpaintDelogo header:: "Version 2.05 : 2022/04/04. Masktools2 v2.2.30 or later is required."
Yes, can only use 64 bit apps with 64 bit Avisynth+ and 64 bit dll's.
Sorry, I'm not sure how subs are extracted, maybe there is text file produced.
(I assisted in the subs extract, but dont remember that part)

EDIT: I've only tried subs detect on short clips, on full movie will likely take a long time.
I presume will extract in '*.srt' text format.

[eXtremeDevil]

Haha don't worry then, maybe other user could help me. I have all x86 versions of the libraries, otherwise the error would be X.dll missing or something like that.

[eXtremeDevil]

OK, I got this:

https://i.ibb.co/Sw68BLkZ/cap.png (https://imgbb.com/)

But now, how to actually get subtitle file/files??

[VoodooFX]

I presume will extract in '*.srt' text format.

*.png files by default.

OK, I got this:

16 subs and 6066 trash - that's because you didn't tuned any dynmask options.

But now, how to actually get subtitle file/files??

First you need to extract proper images of subs, then OCR them.

[eXtremeDevil]

16 subs and 6066 trash - that's because you didn't tuned any dynmask options.

How can I do that?

First you need to extract proper images of subs, then OCR them.

Yes, but how exactly am I supposed to extract them?

[VoodooFX]

How can I do that?

Don't extract blindly with "Extract=1", then read about DynMask in the help at the top of the script, plenty of that info is in the forum thread too.
Can't advice you more without the video.


Yes, but how exactly am I supposed to extract them?

Same as you did in your previous post.

[eXtremeDevil]

Don't extract blindly with "Extract=1", then read about DynMask in the help at the top of the script, plenty of that info is in the forum thread too.
Can't advice you more without the video.



Same as you did in your previous post.

OK, I got to this:

https://i.ibb.co/9mY3PsP8/cap.png (https://ibb.co/9mY3PsP8)

But how to actually dump images to the folder? What do I do with the avs file? How to avs -> png(s)?? I'm completely lost here. Also, can SubtitleEdit read just a bunch of images to perform an OCR? What about timings?

[VoodooFX]

OK, I got to this:

https://i.ibb.co/9mY3PsP8/cap.png (https://ibb.co/9mY3PsP8)

At least this single image looks OK. Usually, using only a mask by color is not good enough.


But how to actually dump images to the folder? What do I do with the avs file? How to avs -> png(s)?? I'm completely lost here.

Use "Extract" to extract images. Just refresh/preview avs for it to do what you commanded.


Also, can SubtitleEdit read just a bunch of images to perform an OCR? What about timings?

Yes, SE can do this and timings too.

[StainlessS]

@VFX, nice job on that eXtremeDevil image :)

[eXtremeDevil]

OK, I got it now, thanks for everything. Just one thing though:

Usually, using only a mask by color is not good enough.

What do you mean with this? Shouldn't all subs be the same color? How to apply a second color/mask/color mask? Just another InpaintDelogo line?

[VoodooFX]

What do you mean with this?

I mean that it's not good enough to isolate subtitles, especially when your selected color is not color, it's grey at 201 level + 15 tol, you would get better by simple brightness mask.

Shouldn't all subs be the same color? How to apply a second color/mask/color mask?

But non subs areas can have your selected color too.
DynMask 2/3 are mix of other masks if those are activated by the settings, for example in your current parameters if you change DynMask to 2/3 then it would use that color mask you set and can use other masks too.

@VFX, nice job on that eXtremeDevil image :)

[*VFX blinks two times*]

[eXtremeDevil]

I mean that it's not good enough to isolate subtitles, especially when your selected color is not color, it's grey at 201 level + 15 tol, you would get better by simple brightness mask.

Well, the sub in this case was grey, not pure white.



But non subs areas can have your selected color too.

But if a sub is white and other area is the same white, how are we supposed to prevent those area selections?

DynMask 2/3 are mix of other masks if those are activated by the settings, for example in your current parameters if you change DynMask to 2/3 then it would use that color mask you set and can use other masks too.

Sorry but I'm not following you. If a sub has only one color, why would I want to filter two colors? Unless you're talking about the sub borders...

[VoodooFX]

Sorry but I'm not following you. If a sub has only one color, why would I want to filter two colors? Unless you're talking about the sub borders...

I didn't say anything about two colors , I said other [B]masks, you can combo them, for example DynMask6 (mask by brightness) and DynMask7 (mask by gradient magnitude map), DynMask4 doesn't mix as I remember.
Basically, the combo mask will contain the mask (aka white pixels) which is present in all mixed masks. [aka if there is black pixel in any mask then it will black in the combo mask]


But if a sub is white and other area is the same white, how are we supposed to prevent those area selections?

There are various methods, only you can know what to use in your case, we don't have crystal ball here. Maybe cut and share a few minutes sample where subs are in brightest background.

[eXtremeDevil]

I didn't say anything about two colors , I said other [B]masks, you can combo them, for example DynMask6 (mask by brightness) and DynMask7 (mask by gradient magnitude map), DynMask4 doesn't mix as I remember.
Basically, the combo mask will contain the mask (aka white pixels) which is present in all mixed masks. [aka if there is black pixel in any mask then it will black in the combo mask]

I get it now.


There are various methods, only you can know what to use in your case, we don't have crystal ball here. Maybe cut and share a few minutes sample where subs are in brightest background.

Of course not, I was just asking about the possibilities and parameters in general, not in my specific case.

Thanks.