InpaintDelogo - advanced logo removal & hardcoded subtitles extraction

[puregreen]

Wow, that's perfect Thank you!

Originally Posted by VoodooFX

Did you re-encoded those videos in your posts?

No, they are untouched.

[puregreen]

What if I created a deblending mask and want to use it on a video where the logo is the same but shifted a little bit?

For example, here's my base mask: https://filetransfer.io/data-package/QxGcwH8N#link

Here's a video I used to generate deblending mask from: https://filetransfer.io/data-package/mjjRj0qE#link

InpaintDelogo( mask="C:\LogoMask596x336p.png", Loc="136,194,-136,-88", Analyze=3, FrB=120, FrW=0, FrS=1, Deep=1, Interp=0, dPP=0, Mode="Deblend")

And let's say I want to use it on this video: https://filetransfer.io/data-package/dzWZCyxL#link

But the logo here is shifted by about 6 px down so the same deblending mask doesn't quite work... Is there a way to make it work? Or the only way is to make a deblending mask from a video where the logo is located at the exact same position?

[VoodooFX]

Originally Posted by puregreen

What if I created a deblending mask and want to use it on a video where the logo is the same but shifted...

1) What is this mask, and it doesn't even fit in your selected Loc, read in the manual about Loc parameter:


2) "Analyze=3, FrB=120, FrW=0, FrS=1" <- manual frame selection doesn't work with positive Analyze.


So, select proper Loc, create proper basemask and deblend masks with 1053334574 clip:

InpaintDelogo( mask="C:\1053334574.bmp", Loc="114,174,-112,-68", Analyze=-3, FrB=76, FrW=227, FrS=3, Deep=1, Mode="Deblend", Automask=1, aMix=0)

In 1053531500 clip the logo is shifted exactly 4 pixels up, so edit the basemask file with Photoshop and move logo up.
A/C deblend masks in ebmp are not stored in full frame dimension, only Loc area, so if you keep dimensions of the Loc area the same then deblend masks fit (they are 'moving' when you change Loc).
[If logo shifted by non-mod2 distance, then you would need to shift that one pixel inside ebmp]

About Loc. You can use "StainlessS"s coords, they are easier to grasp when you are 'moving' area inside frame.
"114,174,-112,-68" would be "114,174,370,94".
So changing "114,174,370,94" -> "114,170,370,94" would move area 4 pixels up.
Remember that "StainlessS"s coords in ebmp filename are converted back to "VoodooFX" coords format [actually they are "VDub" coords]

A/C deblend masks ebmp file is seen and reused if filename corresponds to the parameters in use.
If you don't get idea behind how ebmp file is named just by looking at it then maybe this will help:
https://forum.doom9.org/showthread.p...76#post1948676

RESULTS:

InpaintDelogo( mask="C:\1053334574.bmp", Loc="114,174,-112,-68", Analyze=-3, FrB=76, FrW=227, FrS=3, Deep=1, Interp=0, dPP=0, Mode="Deblend")

Before:

After:

InpaintDelogo( mask="C:\1053531500.bmp", Loc="114,170,370,94", Analyze=-3, FrB=76, FrW=227, FrS=3, Deep=1, Interp=0, dPP=0, Mode="Deblend")

Before:

After:


In 1053531500 clip the logo is damaged by compression on some frames and some remnants of logo are there, you can try "dPP" and "Interp" parameters, they are for hiding artifacts, but they have their own side effects.

[VoodooFX]

Thanks to the new feature in Masktools2 v2.2.30 by @pinterf there is no more glitches with "DynMask=4" on SD videos.

# Version 2.05 : 2022/04/04.  Masktools2 v2.2.30 or later is required.
#                             Tweaked: "DynMask=4" and positive "DynMaskUp", should be a bit faster and no more glitches.
#                             New: "DynLocUp" [Experimental WIP parameter for tests].

[Guilbert]

Hi,

I can't find the correct settings to get proper subtitles with this video: https://mega.nz/file/sgMDyThQ#B0l4VO...3ZcLWmgWBQElpQ (sample)

For now, my script looks like that:

LWLibavVideoSource("")
InpaintDelogo(Loc="200,570,-200,-60", Show=4, DynMask=3, DynTune=218, DynMask4H=200, DynMaskUp=2, Extract=1, ImgDir="")

BTW, I use the latest version of InpaintDelogo (2.09).

Thank you for your help!

[VoodooFX]

Originally Posted by Guilbert

Hi,

I can't find the correct settings to get proper subtitles with this video: https://mega.nz/file/sgMDyThQ#B0l4VO...3ZcLWmgWBQElpQ (sample)

Is your sample a record/re-encode of original video? If yes then cut a sample from original video, and longer, maybe ~15 mins.

[manono]

Hi,

What is the last version for AviSynth 2.60? Is it Version 1.48? And where can I get it?

[VoodooFX]

Originally Posted by manono

Hi,

What is the last version for AviSynth 2.60? Is it Version 1.48? And where can I get it?

Last compatible with AvS 2.6 is v1.48 (would need to be imported with GImport), GImport not needed for v1.33, but "Show=1" is broken there.
Actually, if you just want to inpaint some logo then all versions are good enough.

You can download from GitHub, press "InpaintDelogo.avsi" there, then "History", you'll see different versions, a second button from the right side opens 'that' version.

[VoodooFX]

# Version 2.10 : 2022/05/02.  Added auto generation of the dynamic mask by color (aka "DynMask=5"/"DynColor").
#                             New: "DynColor", "DynColorTol", "DynColorInf", "Show=10".
#                             This dynamic mask by color can pre-patch video for other "DynMask" methods.

Thread that prompted this update, and video samples: Remove timestamp.

Few screens for comparisons:
1) Original
2) DynMask2
3) DynMask3 [Actually, DynMask3 could generate an accurate mask, except for the last digit, read in the link above]
4) DynMask5
5) DynMask2 pre-patched with DynMask5
6) DynMask3 pre-patched with DynMask5
7) Result of the mask from 6th row.

v=LWLibAvVideoSource("D:\Pachacamac E.MP4")
x=v.crop(4,990,-1380,-4)

o1=v.InpaintDelogo(Loc="4,990,-1380,-4", Show=4, DynMask=2, DynTune=150)
o2=v.InpaintDelogo(Loc="4,990,-1380,-4", Show=4, DynMask=3, DynTune=150, Dyn3Seq=54)
o3=v.InpaintDelogo(Loc="4,990,-1380,-4", Show=4, DynMask=5, DynColor=$FFB900)
o4=v.InpaintDelogo(Loc="4,990,-1380,-4", Show=4, DynMask=2, DynTune=150, DynColor=$FFB900)
o5=v.InpaintDelogo(Loc="4,990,-1380,-4", Show=4, DynMask=3, DynTune=150, DynColor=$FFB900, Dyn3Seq=54)
o6=v.InpaintDelogo(Loc="4,990,-1380,-4", Show=0, DynMask=3, DynTune=150, DynColor=$FFB900, Dyn3Seq=54, DynInflate=2).crop(4,990,-1380,-4)

Stackvertical(
\ x,
\ o1.ConvertToYV12().AddBorders(0,2,0,0,$F0F080),
\ o2.ConvertToYV12().AddBorders(0,2,0,0,$F0F080),
\ o3.ConvertToYV12().AddBorders(0,2,0,0,$F0F080),
\ o4.ConvertToYV12().AddBorders(0,2,0,0,$F0F080),
\ o5.ConvertToYV12().AddBorders(0,2,0,0,$F0F080),
\ o6.ConvertToYV12().AddBorders(0,2,0,0,$F0F080))

PS:
@Guilbert, I didn't forgot you. I'll check your video today.

[Selur]

Nice. (this and missing a stable RemoveDirt are the only real reasons I sill have Avisynth on my system didn't switch completely to Vapoursynth)

[VoodooFX]

Originally Posted by Guilbert

Hi,
I can't find the correct settings to get proper subtitles with this video: https://mega.nz/file/sgMDyThQ#B0l4VO...3ZcLWmgWBQElpQ (sample)

For now, my script looks like that:

InpaintDelogo(Loc="200,570,-200,-60", Show=4, DynMask=3, DynTune=218, DynMask4H=200, DynMaskUp=2, Extract=1, ImgDir="")

I don't see much problems in your sample sequence, just lower DynTune to 190 to make extracted subs better.
But I see problems at other places [I got full video in PM].

About your parameters:
DynMask4H is only for DynMask=4, so it does nothing here.
DynMaskUp was meant for HD videos and DynMask=4, for SD videos or small subs use DynLocUp.

Halo around subs is too faint to make it useful, later I'll check if anything can be done about it.

By looking at sequence you shared, I came up with these parameters [dunno if it's worse or better at other places, let me know how it's]:

InpaintDelogo(Loc="300,560,-300,-50", Show=4, DynMask=3, DynTune=190, Dyn3Seq=25, KillBlobs=5, KillNoise=5, RePass=1, DynLocUp=2)

Subs extractor defaults are tuned for a good mask, where mask doesn't have so many artifacts.
If mask has many artifacts then you can:
Lower CorrTh, to ~0.65 - 0.75 [would be less wrong splits]
Lower SubTune, to 200 - 220.
Lower SubMinDur [so wrong splits would be extracted]
Use SubSuspect.


EDIT:

I tried VSF for comparison, sadly it fails, so no incentive to do better, maybe ~perfect mask is not possible from these subs...

On your sample sequence (38265 - 38861) VSF fails a lot, where InpaintDelogo has no problems. One extracted sub for example:

VSF:

InpaintDelogo :



On where I spotted "problems at other places" (6752 - 7269) both fails a lot. One extracted sub for example:

VSF:

InpaintDelogo :

[manono]

Originally Posted by VoodooFX

You can download from GitHub, press "InpaintDelogo.avsi" there, then "History", you'll see different versions, a second button from the right side opens 'that' version.

Thanks a lot! I tried some of the more recent versions first but couldn't get them going. Then I downloaded V. 1.48 and set it up. I used "Inpaint" Mode to get rid of the logo but, of course, it left behind the blurs during moving backgrounds.

Then, based on your tips further up the page, I decided to try to get the "Both" Mode going as the logo wasn't completely opaque. And ... it worked! No sign at all of the logo any longer. I'm very pleased and have you to thank for it, VoodooFX.

[VoodooFX]

Originally Posted by manono

Thanks a lot! I tried some of the more recent versions first but couldn't get them going. Then I downloaded V. 1.48 and set it up. I used "Inpaint" Mode to get rid of the logo but, of course, it left behind the blurs during moving backgrounds.

Then, based on your tips further up the page, I decided to try to get the "Both" Mode going as the logo wasn't completely opaque. And ... it worked! No sign at all of the logo any longer. I'm very pleased and have you to thank for it, VoodooFX.

Finally you moved to proper logo deblending from simple inpaintings. [or was it some other guy?]

So it wasn't AvS+ issue? Why didn't you ask for help then?
For not Inpaint mode you definitely want to use latest version, other modes progressed a lot.

You should use Deblend for non-opaque logos, Both is meant for other things and can have unpredictable side effects if you use it instead of Deblend.

Post issues you had, full avs script, frame sample from video and we'll solve it.

[Guilbert]

Thanks a lot, VoodooFX!

(I'll test these parameters next weekend and keep you in touch.)

[VoodooFX]

Originally Posted by Guilbert

Thanks a lot, VoodooFX!
(I'll test these parameters next week-end and keep you in touch.)

I quick looked what can be done with tcanny edge masking, I think it can replace positive KillBlobs and RePass, and it's faster.

KillBlobs and RePass:

new tcanny stuff:


And in general there are less bad splits and less some other artifacts with tcanny.

EDIT:
Need to investigate what Dogway's masks are doing...
Maybe someone knows magic already how to isolate letters inside this halo precisely [from the sample below]?

[VoodooFX]

Originally Posted by Guilbert

...

Masking with "new tcanny stuff": 204 FPS
Masking with KillBlobs/RePass: 91 FPS

Dunno when I'll have time to release a new version, so here is the mask: https://we.tl/t-MjgKuwhWWn

Used Dyn3Seq=24, [I checked and found shortest sub is 24 frames]

Took ~7 mins to encode the mask, and ~3 mins to extract images with: SubsMask2Img(ImgDir="D:\test", SubMinDur=6, SubSuspect=23)

[Guilbert]

Wow, that's a nice improvement!

I use these parameters for this show, now:

InpaintDelogo(Loc="300,560,-300,-50", Show=4, DynMask=3, DynTune=190, Dyn3Seq=24, KillBlobs=5, KillNoise=5, RePass=1, DynLocUp=2, CorrTh=0.75, SubTune=220)

It took me only 15-20 minutes to OCR and edit with Tesseract 5.0.1. Can't wait to test the next release with the "new tcanny stuff", then.

[VoodooFX]

# Version 2.12 : 2022/05/14.  Added generation of the dynamic mask by gradient magnitude map (aka "DynMask=7"/"DynTEdge").
#                             New: "DynTEdge", "DynTEdgeThY", "DynTEdgeInf", "DynTuneInf", Show=10, Show=11.
#                             vsTEdgeMask plugin is required.
# Version 2.11 : 2022/05/13.  Rewrite: "DynColor" ~30%-300% faster. New: "DynMask=6/"DynTune". Removed: "Show=10".

@Guilbert
I used these parameters to get the mask in my previous post:
InpaintDelogo(Loc="300,560,-300,-50", Show=4, DynMask=3, Dyn3Seq=24, DynTune=190, DynTEdge=1, DynTEdgeThY=20, DynTEdgeInf=2, DynLocUp=2)

[VoodooFX]

Improved subs extraction: ~90% reduction of non-sub images.

# Version 2.13 : 2022/05/21.  SubsMask2Img: Added average luma check to catch non-sub images.
#                             New: "SubMinLuma", "SubSusLuma".
#                             Renamed: "SubSuspect" to "SubSusDur".

[VoodooFX]

# Version 2.14 : 2022/05/22.  SubsMask2Img: New option to extract raw non-binarized images.
#                             Enabled by "Extract=2" in InpaintDelogo() or "Raw" in SubsMask2Img().
#                             New: "Raw", "RawAvr".

Should be better than VSF's raw images, as I added averaging for raw images, that should increase OCR accuracy [it's enabled by default].

For example averaged raw sub image:


Not averaged raw sub image:




The feature request:

Chinese-like characters are not well supported by tesseract or any other softwares. It would be nice to use paid Baidu OCR api on those languages. Since it's an enterprise solution, they can handle hardsub detection a lot better than custom masks, more often than not, raw image will yield better results.

He sent me few examples of Baidu OCR:

Sub image: https://i.imgur.com/goxj4KB.png



Sub image: https://i.imgur.com/u3nzosV.png [not raw, just to check how it's better than Tesseract]



Note: Google Cloud Vision OCR should be something similar to Baidu OCR.