InpaintDelogo - advanced logo removal & hardcoded subtitles extraction

[VoodooFX]

Thank you, your script works perfect for characters with halo, but for English sub without halo, the characters often miss. Which mask do you recommend? Or is there a well-adjusted setting?
This is my setting for now, It seems fine, does it need any changes? Dyntune seems to be the key parameter to keep character complete, is there other parameter that I can use?
\ DynMask=4, DynTune=200,
\ DynMask4H=110,
\Extract=1, Show=4,
Where is sample of your video?
For different video/subtitles you need to set different thresholds, Dynmask4 is meant only for subtitles with distinct halo, so you are using wrong settings. There is no "well-adjusted" thresholds to magically fit all subtitles.

[evafans]

Where is sample of your video?
For different video/subtitles you need to set different thresholds, Dynmask4 is meant only for subtitles with distinct halo, so you are using wrong settings. There is no "well-adjusted" thresholds to magically fit all subtitles.
https://drive.google.com/file/d/1uWtcp8tYFsIiIGcn99v0RXlyCXVkiZeF/view?usp=sharing

[StainlessS]

@ evafans,

Dynmask4 is meant only for subtitles with distinct halo
The halo is key to making the subtitle detections.

You have to have or create a google account to access your link.

[evafans]

https://wormhole.app/vD0Mq#_yqBj3yogM4rCHQYPHDIjw

[VoodooFX]

https://wormhole.app/vD0Mq#_yqBj3yogM4rCHQYPHDIjw

You can't get simpler subs like these, subs are brighter than anything else in this video/sample, so if whole video is like that then you can use simplest and fastest dynamic mask AKA DynMask6:


LWLibAvVideoSource("D:\sample.mkv")
InpaintDelogo(Loc="400,840,-400,-80", Show=4, DynMask=6, DynTune=215)


Inside the script [InpaintDelogo.avsi], at the top there is manual/help about parameters, you can open it with any text editor like Notepad++ or PSPad.
Come back with more challenging sample/subs, for example where you get artifacts. :p

[VoodooFX]

The halo is key to making the subtitle detections.
Halo is the key only in DynMask4, there is other options for DynMask where halo is not taken into account.

[kedautinh12]

Hi VX, i tried your script but can use cause it's hard to got mask logo.bmp and use your script?? Can you got logo.bmp from there video and guide me to use your script, thanks
M-On: https://drive.google.com/file/d/1Zsx7OYSGWL6W3BsJJn8fWPMkdd-EgdQE/view?usp=sharing
SSTV: https://drive.google.com/file/d/1ntJYRiwL8thnUDLlFZe_oTdM3zrxe6rR/view?usp=sharing
SSTV+: https://drive.google.com/file/d/1i3jj5thtLpYP-YG-Xm-5apAsaGXez2YI/view?usp=sharing

[VoodooFX]

Hi VX, i tried your script but can use cause it's hard to got mask logo.bmp and use your script?? Can you got logo.bmp from there video and guide me to use your script, thanks
Those links doesn't work for me, use normal filelocker like Wetransfer or Mega ect., you can share from Google Drive too.

And share the script where you are trying to get that mask.

[kedautinh12]

Those links doesn't work for me, use normal filelocker like Wetransfer or Mega ect., you can share from Google Drive too.

And share the script where you are trying to get that mask.

Ok, i edited my link and about script, i use loc and mask. Other parameter is default

[VoodooFX]

i tried your script... it's hard... guide me to use your script
and about script, i use loc and mask. Other parameter is default
I see that you didn't tried anything, first step in guide is to read it...

So you've used 2 parameters so far... even in "Basic short guide" on the first post are mentioned 5 parameters, after you read it - post what you didn't understand.
Meanwhile, I'll look at your first link.

[VoodooFX]

M-On: https://drive.google.com/file/d/1Zsx7OYSGWL6W3BsJJn8fWPMkdd-EgdQE/view?usp=sharing

Video is interlaced, so first deinterlace it to lossless file.
Logo is transparent = use Deblend mode.
Trim to exclude parts where there is no logo or logo is animated.
Run the script posted below with Automask=1 to create "bmp" basemask, then run with 0 to create deblending masks and delogo.
Animated parts you can delogo with Inpaint mode.


LWLibAvVideoSource("D:\EONIAN_deinterlaced.mkv")
Trim(61,10907)
InpaintDelogo( mask="D:\mask2.bmp", Automask=0, aMix=1, Loc="1252,878,-90,-64", Mode="Deblend")


Similarly you can remove logo at the left side too.

Logo:
https://i.imgur.com/eTxsNe4.jpeg
https://i.imgur.com/FOmKVDG.jpeg

Delogo:
https://i.imgur.com/HZNQF8f.jpeg
https://i.imgur.com/jBIxngM.jpeg

[ChaosKing]

Similarly you can remove logo at the left side too.


You mean the other left, also know as right.
I was staring for half a minute at the left text and couldn't find any difference:devil:

[VoodooFX]

You mean the other left, also know as right.
I was staring for half a minute at the left text and couldn't find any difference:devil:
Left side is left, not idea what you meant by "right". :confused:
I meant that he can delogo the text at the left side himself. :)

[kedautinh12]

I see that you didn't tried anything, first step in guide is to read it...

So you've used 2 parameters so far... even in "Basic short guide" on the first post are mentioned 5 parameters, after you read it - post what you didn't understand.
Meanwhile, I'll look at your first link.

Oh, i tried to understand but your script hard to understand, sr i'm noob. And my result is video with green color cover all video:D

[kedautinh12]

Video is interlaced, so first deinterlace it to lossless file.
Logo is transparent = use Deblend mode.
Trim to exclude parts where there is no logo or logo is animated.
Run the script posted below with Automask=1 to create "bmp" basemask, then run with 0 to create deblending masks and delogo.
Animated parts you can delogo with Inpaint mode.


LWLibAvVideoSource("D:\EONIAN_deinterlaced.mkv")
Trim(61,10907)
InpaintDelogo( mask="D:\mask2.bmp", Automask=0, aMix=1, Loc="1252,878,-90,-64", Mode="Deblend")


Similarly you can remove logo at the left side too.

Logo:
https://i.imgur.com/eTxsNe4.jpeg
https://i.imgur.com/FOmKVDG.jpeg

Delogo:
https://i.imgur.com/HZNQF8f.jpeg
https://i.imgur.com/jBIxngM.jpeg

Like you said. I use this script
LWLibAvVideoSource("D:\EONIAN_deinterlaced.mkv")
Trim(61,10907)
InpaintDelogo( mask="D:\mask2.bmp", Automask=1, aMix=1, Loc="1252,878,-90,-64", Mode="Deblend") # to make .bmp file
InpaintDelogo( mask="D:\mask2.bmp", Automask=0, aMix=1, Loc="1252,878,-90,-64", Mode="Deblend") # to Delogo

Right?

[VoodooFX]

Right?
No, you don't stack it like that. Run with Automask enabled to get the basemask, then run with Automask disabled to do delogo.

[kedautinh12]

No, you don't stack it like that. Run with Automask enabled to get the basemask, then run with Automask disabled to do delogo.

But i only use one pass encode. And my scripts i use both. Automask=1 on top and automask=0 on bottom

[VoodooFX]

"Run" doesn't mean encode, that means you do preview of your script.
And that script is not ready to encode, you need to join parts with no logo, and delogo and join the animated parts.

[kedautinh12]

"Run" doesn't mean encode, that means you do preview of your script.
And that script is not ready to encode, you need to join parts with no logo, and delogo and join the animated parts.

I know what you say, but i ask this script correct in delogo part??

[VoodooFX]

If you understand then I don't understand what you are asking.

[kedautinh12]

If you understand then I don't understand what you are asking.

You said: "join parts with no logo, and delogo and join the animated parts". And i ask my scripts above correct with "delogo part" as you said

[VoodooFX]

You said: "join parts with no logo, and delogo and join the animated parts". And i ask my scripts above correct with "delogo part" as you said
Sorry, I don't understand what you write.

[kedautinh12]

Sorry, I don't understand what you write.

Lol, just forget about it :D

[pcroland]

If anyone is interested I created a small script that can extract forced cues from a full subtitle based on the folder that SubsMask2Img created:
https://github.com/pcroland/substoforced
❯ ./stf.py -h
substoforced 1.0.0

USAGE: stf.py [-h] [-v] [-s SUB] [-f FOLDER]

FLAGS:
-h, --help show this help message.
-v, --version show version.
-s, --sub SUB specifies srt input
-f, --folder FOLDER specifies a folder where SubsMask2Img generated timecodes (optional)
you should remove the junk from there manually

[VoodooFX]

If anyone is interested I created a small script that can extract forced cues from a full subtitle based on the folder that SubsMask2Img created:
https://github.com/pcroland/substoforced

Could you describe more specifically what it does?

[pcroland]

It keeps the cues from the full subtitle that overlaps with any of the timecodes generated by SubsMask2Img. Lets say you have an English movie with French parts that are hardsubbed and you have a dub in another language where French parts aren't translated and you only have a full subtitle for it. You can generate the forced automatically for it.
Another use case would be to have a video with hardsubs that are also present in the full subtitle. You could create a subtitle based on your full subtitle without OCRing image (so it has less errors).

[VoodooFX]

Now it's clear. I was thinking what you are extracting there when you have srt already. :)

[Emulgator]

First off, many thanks to you, VoodooFX for building and maintaining that huge and useful script.

A suggestion: While having to handle plain white text logo with black borders (subtitle style) I came across the need to give expand=3.
expand=2 would still leave inpainting artifacts, which postprocessing could mend only a bit.
Going into script version 2.17 line 1178 it was well possible to change the assert line to acommodate expand up to 5 without breaking the underlying function balloonPX.
buginflate = (inflate < 0 || inflate > 2) ? true : false
changed to
buginflate = (inflate < 0 || inflate > 5) ? true : false

Maybe worth a suggestion to have that in the coming version.

[VoodooFX]

@Emulgator

Thing is that I never saw a practical case where I would need it more than 2...
You would need to share your logo case so I can see it with my own dead eyes to change my mind on that option.

Btw, you are not the first one asking this, and all of them were doing something wrong and wanted to "expand" stuff more to cover mistakes.

[Emulgator]

It was a subtitle style logo. White body, plus black borders, plus a bit halo.
Photosobbing -> Curves -> Isolating the white body left my mask without the black borders, let alone the faint halo, so too thin
To get that inpainted properly it would either take photosobbing again to pad up the lost borders and halo, or take the borders into account first.

Well, more work...or simply hand that padding job over to Inpaint Delogo, which did work nicely.

I guess the old fat bordered Sony Handycam style time code rendering would benefit from that too, still got to capture a case to publish.
Badly needed for stabilising such footage, because stabilising alone leaves you with shaky timecode, so timecode got to be removed first before stabilising.

[VoodooFX]

Photosobbing -> Curves -> Isolating the white body left my mask without the black borders
So it's problem of you not making a proper basemask with Photozhop. If logo has black borders then select a frame with a bright background for edits. Use Magic Wand [there is Contiguous option] and Paint Bucket.
Or just use Automask.

I guess the old fat bordered Sony Handycam style time code rendering would benefit from that too
Inflate is only for static logos, for a dynamic mask there is DynInflate, it's limited to "4".

[Emulgator]

Good, I will try that.
Automask got it wrong, no pure logo on black to be had, so I would have liked to tune thresholds and such, but at the moment I am only a Delogo beginner...

[VoodooFX]

Did you tried to use aMix option with Automask? Are you sure logo is not transparent?
Without video I can't tell you much, share it by PM if you want.

[Guilbert]

Hi, @VoodooFX.

Can you please take a look to the video I send you?
I can't get a good result with InpaintDelogo to OCR these hardcoded subtitles.

I tried with these parameters, with no luck:
InpaintDelogo(Loc="280,560,-280,-40", Show=4, DynMask=4, DynTune=210, DynMask4H=60, DynMaskUp=2)

Thank you for your help! :)

[VoodooFX]

I tried with these parameters, with no luck:
InpaintDelogo(Loc="280,560,-280,-40", Show=4, DynMask=4, DynTune=210, DynMask4H=60, DynMaskUp=2)

Did you read the manual how to set those parameters? In this video a text brightness is way over "210" and a halo brightness as "60" is way too low.

DynTune aka text brightness - set high as possible [till text doesn't disappear].
DynMask4H aka halo brightness - set low as possible [till letters doesn't disappear].
KillBlobs - set low as possible [till letters doesn't disappear, good indicator are capital "M" and "W" letters].


v=LWLibAvVideoSource("D:\video.1080p.mkv")
x=v.crop(360,820,-360,-80)

o1=v.InpaintDelogo(Loc="360,820,-360,-80", Show=4, DynMask=4, DynTune=210, DynMask4H=60)
o2=v.InpaintDelogo(Loc="360,820,-360,-80", Show=4, DynMask=6, DynTune=240)
o3=v.InpaintDelogo(Loc="360,820,-360,-80", Show=4, DynMask=4, DynTune=240, DynMask4H=140)
o4=v.InpaintDelogo(Loc="360,820,-360,-80", Show=4, DynMask=4, DynTune=240, DynMask4H=140, KillBlobs=-4, KillNoise=-5)

Stackvertical(
\ x,
\ o1.ConvertToYV12().AddBorders(0,2,0,0,$F0F080).Subtitle("1: your settings"),
\ o2.ConvertToYV12().AddBorders(0,2,0,0,$F0F080).Subtitle("2: only DynTune"),
\ o3.ConvertToYV12().AddBorders(0,2,0,0,$F0F080).Subtitle("3: DynMask=4"),
\ o4.ConvertToYV12().AddBorders(0,2,0,0,$F0F080).Subtitle("4: DynMask=4 + KillBlobs & KillNoise"))


Frame 12270:

https://thumbs2.imgbox.com/26/87/YlWDrkXT_t.png (https://images2.imgbox.com/26/87/YlWDrkXT_o.png)


Here is the extracted result ["00_08_09_640__00_08_12_360.png"]:

InpaintDelogo(Loc="360,820,-360,-80", Show=4, DynMask=4, DynTune=240, DynMask4H=140, KillBlobs=-4, KillNoise=-5, DynMaskUp=2, Extract=1, ImgDir="D:\Images", ImgInflate=1)


https://thumbs2.imgbox.com/59/82/g6jh9jO4_t.png (https://images2.imgbox.com/59/82/g6jh9jO4_o.png)

[I]ImgInflate=1 because DynTune=240 is a bit too high for some sub lines [there are some small "holes" in text], so ImgInflate=1 fills them in.

[Guilbert]

No, I didn't read the manual yet, but I will, thank you for the advice.
Thank you as well for the explanations and those parameters, it looks amazing! :)

Do you know if Google Cloud Vision is able to detect italics, by any chance, please?
(The API was added recently in Subtitle Edit. ;))

[VoodooFX]

Do you know if Google Cloud Vision is able to detect italics, by any chance, please?
(The API was added recently in Subtitle Edit. ;))
I dunno. You can ask at Subtitle Edit's thread. (https://forum.doom9.org/showthread.php?t=162721)

[Zetti]

I have tested the subtitle extraction function with your settings from post 435 like this:

LoadPlugin("C:\MeGUI-x64\tools\ffms\ffms2.dll")
FFVideoSource("D:\Test\Test.mp4", threads=1)
InpaintDelogo(Loc="164, 362, -150, -38", Show=4, DynMask=4, DynTune=240, DynMask4H=140, KillBlobs=-4, KillNoise=-5, DynMaskUp=2, Extract=1, ImgDir="D:\Test", ImgInflate=1)

So i can see what it is doing with my video test, but i got this result out of it:

https://i.postimg.cc/yY55nsGx/test.jpg

What can i have done wrong?

[VoodooFX]

I have tested the subtitle extraction function with your settings from post 435 like this:
...
What can i have done wrong?
Most likely - subtitles in your video are not the same type as subtitles in Guilbert's video, you need to set parameters fitting your subtitles.
Cut [no re-encode] and post a short few mins sample of video if you want advice [or a whole video by PM].

[Zetti]

Here is small sample: https://www.file.io/9gdT/download/Xm0V7j2kmDMc

[VoodooFX]

Here is small sample: https://www.file.io/9gdT/download/Xm0V7j2kmDMc

Try this:

InpaintDelogo(Loc="130, 358, -130, -34", Show=4, DynLocUp=4, DynMask=4, DynTune=210, DynMask4H=190, KillBlobs=-7, Extract=1, ImgDir="D:\Test", ImgInflate=1, CorrTh=0.7)

[Zetti]

I got a good result, thank you very much.

[Guilbert]

@VoodooFX, what do you think about those parameters, please:

InpaintDelogo(Loc="200,490,-200,-70", Show=4, DynMask=3, Dyn3Seq=32, DynTune=240, DynTEdge=1, DynTEdgeThY=60, DynTEdgeInf=8, DynLocUp=2, KillBlobs=-5, KillNoise=-5, Extract=1, ImgDir="", ImgInflate=1)

(Video sent by PM.)

I find that the font is too altered, which makes the OCR complicated. :/

[VoodooFX]

@VoodooFX, what do you think about those parameters, please
Nonsense parameters, please read the manual about parameters you are using, then ask what you don't understand.
Copy-pasting random parameters from other posts is not how it works.

[Guilbert]

Well, sorry, I try to do my best to learn how to use InpaintDelogo and not to bother you anymore, so I read the manual again carefully, and I started from scratch with those ones:

InpaintDelogo(Loc="200,500,-200,-90", Show=4, DynMask=4, DynMask4H=90, DynLocUp=3, KillBlobs=7, KillNoise=5)

Unlike the parameters above, the font looks much better, but there are still some artifacts and the credits still appear.

Is it possible to improve that, please?

[VoodooFX]

Well, sorry, I try to do my best...
No need to be sorry, just that you are asking for same things, better ask what do you not understand.

so I read the manual again carefully, and I started from scratch
Start from "DynTune" parameter, what do you not understand about it?

Here is your used "DynTune=240" [frame 12349]:

v=LWLibAvVideoSource("D:\Brassic.mp4")
x=v.crop(200,490,-200,-86)

o1=v.InpaintDelogo(Loc="200,490,-200,-86", Show=4, DynMask=6, DynTune=240)

Stackvertical(
\ x,
\ o1.ConvertToYV12().AddBorders(0,2,0,0,$F0F080))


https://images2.imgbox.com/e2/ca/ETBPO75c_o.png



there are still some artifacts and the credits still appear.
If you want an advice on some artifact then you need to share where I can see that artifact, the frame number or timing.
If the credits looks like subtitles then you'll see them in a mask.

[evafans]

Hello，Sir. Could you help me find the right settings for the sample video， This one is half transparent without halo. I found it very tricky.
https://wormhole.app/qeyOM#mFSnGFBdgXFbjsXXfTEwLg

[VoodooFX]

Hello，Sir. Could you help me find the right settings for the sample video， This one is half transparent without halo. I found it very tricky.
https://wormhole.app/qeyOM#mFSnGFBdgXFbjsXXfTEwLg
Where is your full script?

These subs are not very distinctive from the background and I think they are transparent, but nothing too complicated.
Mix few masks like Dynmask6 with Dynmask7, maybe this will help you to understand:


v=LWLibAvVideoSource("D:\test.mp4")
x=v.crop(130,950,-130,-50)
o1=v.InpaintDelogo(Loc="130,950,-130,-50", Show=4, DynLocUp=2, DynMask=6, DynTune=190)
o2=v.InpaintDelogo(Loc="130,950,-130,-50", Show=4, DynLocUp=2, DynMask=7, DynTEdge=1, DynTEdgeThY=30, DynTEdgeInf=4)
o3=v.InpaintDelogo(Loc="130,950,-130,-50", Show=4, DynLocUp=2, DynMask=2, DynTune=190, DynTEdge=1, DynTEdgeThY=30, DynTEdgeInf=4, KillNoise=5)

Stackvertical(
\ x.Spline16Resize(2180,160).Subtitle("0: Original", size=40),
\ o1.ConvertToYV12().AddBorders(0,2,0,0,$F0F080).Subtitle("1: DynTune", size=40),
\ o2.ConvertToYV12().AddBorders(0,2,0,0,$F0F080).Subtitle("2: DynTEdge", size=40),
\ o3.ConvertToYV12().AddBorders(0,2,0,0,$F0F080).Subtitle("3: Non-temporal mix of 1 & 2", size=40))


Frame 3445:
https://thumbs2.imgbox.com/d9/e0/U2zxom0f_t.png (https://images2.imgbox.com/d9/e0/U2zxom0f_o.png)

Here is the extracted result ["00_02_23_227__00_02_26_271.png"]:

InpaintDelogo(Loc="130,950,-130,-50", Show=4, DynLocUp=2, DynMask=2, DynTune=190, DynTEdge=1, DynTEdgeThY=30, DynTEdgeInf=4, KillNoise=5, Extract=1, ImgDir="D:\Images", CorrTh=0.7, ImgInflate=1)

https://thumbs2.imgbox.com/17/09/c6LcCR5O_t.png (https://images2.imgbox.com/17/09/c6LcCR5O_o.png)

https://images2.imgbox.com/cc/11/drBNcJSB_o.png

If full video has more than one line subs then set Loc accordingly.
What do you use to OCR these subs?

[VoodooFX]

I noticed one a different color subtitle (translates writing on the door), these won't be detected by the settings in the previous post [frame 15063]:

https://thumbs2.imgbox.com/d5/c1/G1BUROgF_t.png (https://images2.imgbox.com/d5/c1/G1BUROgF_o.png)

Here are quick settings to conform that one subtitle [no idea if it will catch other such subtitles]:


InpaintDelogo(Loc="130,950,-130,-50", Show=4, DynLocUp=2, DynMask=3, Dyn3Seq=40, DynTune="130 - 160", DynTEdge=1, DynTEdgeThY=30, DynTEdgeInf=4, KillNoise=5, Extract=1, ImgDir="D:\Images", CorrTh=0.7, ImgInflate=1, SubMinDur=20)


Extracted:

https://thumbs2.imgbox.com/fc/3d/sOJnNl0F_t.png (https://images2.imgbox.com/fc/3d/sOJnNl0F_o.png)

There was one more trash subtitle extracted:

https://images2.imgbox.com/91/de/bF1CovqS_o.png

[evafans]

Thanks. I tried Mask 3, 4, 6 on this one, I shall study other masks.