Vine, cutting edge quality de-halo filter and morphology stuff

[feisty2]

readme and more
https://github.com/IFeelBloated/Vine

Vine does not turn your video to a piece of cartoonized shit like other dehalo filters do.
it's slow but the performance is kind of practical still.

[feisty2]

r2:
replaced hacky floatfilters with the ones in core functions
now requires vaporsynth r33 test4 or later

[feisty2]

r3:
1. strict sanity check in every single function, a lot more user friendly and should be ok to work like any typical module now
2. trivial modifications to the mask
3. removed the morphology class and exposed all morphological functions with Vine.Dehalo
4. re-licensed to LGPL v3.0
5. trivial modifications to the doc (color images instead of gray stuff, etc..)
6. gitignore

[hydra3333]

Trying out Vine and Plum but receiving an error message I'm not sure what to do with:
"C:\SOFTWARE\Vapoursynth-x64\VSPipe.exe" "T:\HDTV\WDTVlive\MP4-VS\x64-1\test.mpg.2016.08.24.14.00.23.61.vpy" - --y4m | "C:\SOFTWARE\ffmpeg\0-homebuilt-x64\x264-mp4.exe" - --stdin y4m --thread-input --frames 45810 --profile high --level 4.1 --preset slow --interlaced --tff --no-cabac --crf 16 --sar 64:45 --colormatrix bt470bg -o "s:\HDTV\WDTVlive\MP4-VS\x64-1\test-temp.2016.08.24.14.00.23.61.h264"
Script evaluation failed:
Python exception: Vine.Dehalo: the sample type of src has to be single precision!
Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1491, in vapoursynth.vpy_evaluateScript (src\cython\vapoursynth.c:26905)
File "T:\HDTV\WDTVlive\MP4-VS\x64-1\test.mpg.2016.08.24.14.00.23.61.vpy", line 487, in <module>
main()
File "T:\HDTV\WDTVlive\MP4-VS\x64-1\test.mpg.2016.08.24.14.00.23.61.vpy", line 141, in main
video = Vine.Dehalo(video, radius=[2, None])
File "C:\SOFTWARE\Vapoursynth-x64\Vine.py", line 113, in Dehalo
raise TypeError("Vine.Dehalo: the sample type of src has to be single precision!")
TypeError: Vine.Dehalo: the sample type of src has to be single precision!
Script:
import vapoursynth as vs
import havsfuncTS as haf # this version uses vanilla TemporalSoften instead of TemporalSoften2, as it will be "better" over time
import mvsfunc as mvs # http://forum.doom9.org/showthread.php?t=172564
import finesharp as finesharp # http://forum.doom9.org/showthread.php?p=1777815#post1777815 http://avisynth.nl/index.php/FineSharp
import Plum # http://forum.doom9.org/showthread.php?t=173775 https://github.com/IFeelBloated/Plum
import Vine # http://forum.doom9.org/showthread.php?t=173703 https://github.com/IFeelBloated/Vine
core = vs.get_core(accept_lowercase=True) # leave off threads=8 so it auto-detects threads
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\AddGrain.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\BM3D.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\d2vsource.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\Deblock.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\DFTTest.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\fmtconv.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\KNLMeansCL.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\libawarpsharp2.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\libmvtools.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\libmvtools_sf_emt64.dll') # for Plum, the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\libnnedi3.dll') # the r'' indicates do not treat special characters and accept backslashes
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\scenechange.dll') # http://forum.doom9.org/showthread.php?t=166769
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\TCanny_AVX.dll') # http://forum.doom9.org/showthread.php?t=166769
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\vcfreq.dll') # http://forum.doom9.org/showthread.php?t=171413 http://www.avisynth.nl/users/vcmohan/index.html ... for Plum
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\vsdctfilter.dll') # http://vfrmaniac.fushizen.eu/works
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\vsfft3dfilter.dll') # http://vfrmaniac.fushizen.eu/works
core.std.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\vapoursynth64\plugins\dll-to-choose-from\Yadifmod.dll') # the r'' indicates do not treat special characters and accept backslashes
core.avs.LoadPlugin(r'C:\SOFTWARE\Vapoursynth-x64\DGIndex\DGDecodeNV.dll')
# NOTE:: INDENTING IS CRITICAL AND MUST BE PRECISELY THE SAME WITHIN EACH FUNCTION
# (functions removed in this example)
def main():
video = core.d2v.Source(r'T:\HDTV\WDTVlive\MP4-VS\x64-1\test.d2v')
video = core.std.SeparateFields(video, tff=True)
video = haf.Deblock_QED(video)
video = haf.Weave(video, tff=True)
video = mvs.AssumeTFF(video)
video = haf.QTGMC(video, TFF=True, Preset="Very Slow", Sharpness=1.2, SLMode=2, EZKeepGrain=1.2, NoiseProcess=2) # result is double framerate progressive, so re-interlate it later
video = Vine.Dehalo(video, radius=[2, None])
deconv = Plum.Basic(video)
conv = Plum.Basic(video,mode="convolution")
sup = Plum.Super([video, None])
supdeconv = Plum.Super([video, deconv])
supconv = Plum.Super([conv, None])
video = Plum.Final([video, deconv, conv], [sup, supdeconv, supconv], strength=2.1, cutoff=16) # middlin
video = core.std.SeparateFields(video, tff=True)
video = core.std.SelectEvery(video, cycle=4, offsets=[0, 3]) # looks like avisynth
video = haf.Weave(video, tff=True)
video = mvs.AssumeTFF(video)
video.set_output()
return True
main()

Any and all suggestions welcomed.
... also about "best" positioning of Vine for interlaced material (eg after QTGMC or within the prior separatefields/deblock)

[hydra3333]

Using "video = core.text.ClipInfo(video)" shows that the video sample type incoming into Vine = Integer.
How do I turn it into single precision for use in Vine, and back again for use in other functions ? Would there be quality losses associated with the conversions ?

[feisty2]

video = haf.QTGMC(video, TFF=True, Preset="Very Slow", Sharpness=1.2, SLMode=2, EZKeepGrain=1.2, NoiseProcess=2) # result is double framerate progressive, so re-interlate it later

video = core.fmtc.bitdepth(video,bits=32,fulls=False,fulld=True)

video = Vine.Dehalo(video, radius=[2, None])
deconv = Plum.Basic(video)
conv = Plum.Basic(video,mode="convolution")
sup = Plum.Super([video, None])
supdeconv = Plum.Super([video, deconv])
supconv = Plum.Super([conv, None])
video = Plum.Final([video, deconv, conv], [sup, supdeconv, supconv], strength=2.1, cutoff=16) # middlin
video = core.std.SeparateFields(video, tff=True)

[Mystery Keeper]

Also, here's a snippet for plugins autoloading. It also adds a path from which other scripts can be imported easily.
from __future__ import print_function

import vapoursynth as vs
core = vs.get_core(threads=8)
core.set_max_cache_size(12000)

import sys
sys.path.append('D:\\vapoursynth-plugins\\py\\')
import platform
architecture = platform.architecture()
if architecture[0] == '64bit':
vapoursynth_plugins_path = 'D:\\vapoursynth-plugins\\64bit\\'
else:
vapoursynth_plugins_path = 'D:\\vapoursynth-plugins\\32bit\\'
print('Plugins folder: ', vapoursynth_plugins_path, end='\n', file=sys.stderr)
import os
for filename in os.listdir(vapoursynth_plugins_path):
if filename[-4:] != '.dll':
continue
try:
core.std.LoadPlugin(vapoursynth_plugins_path + filename)
except Exception as e:
print('Error: ', e, end='\n', file=sys.stderr)

[hydra3333]

Thank you.

[feisty2]

r4:
algorithmic upgrade, better quality and a bit slower

[hydra3333]

Hello. With the latest versions of VS plugins and VS R33.1, the sdcript below either
(i) causes vspipe to crash straight away (the windows dialogue popup box type of crash, without any python traceback) or
(ii) the process encodes at 0.35 fps

What am I doing wrong ? ... suggestions very much welcomed.

Input is PAL 576i mpeg2.

"C:\SOFTWARE\Vapoursynth-x64\VSPipe.exe" "T:\HDTV\WDTVlive\MP4-VS\x64-1\test.mpg.2016.09.13.19.17.25.42.vpy" - --y4m | "C:\SOFTWARE\ffmpeg\0-homebuilt-x64\x264-mp4.exe" - --stdin y4m --thread-input --profile high --level 4.1 --preset slow --interlaced --tff --no-cabac --crf 16 --sar 64:45 --colormatrix bt470bg -o "s:\HDTV\WDTVlive\MP4-VS\x64-1\test-temp.2016.09.13.19.17.25.42.h264"

import vapoursynth as vs
import havsfuncTS as haf # this version uses vanilla TemporalSoften instead of TemporalSoften2, as it will be "better" over time
import mvsfunc as mvs # http://forum.doom9.org/showthread.php?t=172564
import finesharp as finesharp # http://forum.doom9.org/showthread.php?p=1777815#post1777815 http://avisynth.nl/index.php/FineSharp
import Plum # http://forum.doom9.org/showthread.php?t=173775 https://github.com/IFeelBloated/Plum
import Vine # http://forum.doom9.org/showthread.php?t=173703 https://github.com/IFeelBloated/Vine
import Oyster # http://forum.doom9.org/showthread.php?t=173470 https://github.com/IFeelBloated/Oyster
import mvmulti # http://forum.doom9.org/showthread.php?p=1777891#post1777891
core = vs.get_core(accept_lowercase=True) # leave off threads=8 so it auto-detects threads
# snip - loaded plugins and stuff
def main():
video = core.d2v.Source(r'T:\HDTV\WDTVlive\MP4-VS\x64-1\test.d2v')
video = core.std.Trim(video,first=0,length=1000)
video = core.std.SeparateFields(video, tff=True)
# Vine only accepts 32bit float input
video = core.fmtc.bitdepth(video,bits=32,fulls=False,fulld=True) #convert to 32bit float. Specify if the clip is full-range (True) or TV-range (False). fulls is for input, fulld for output.
# vine middlin
video = Vine.Dehalo(video, radius=[2, None])
video = core.fmtc.bitdepth(video,bits=8,fulls=True,fulld=False) #convert to 8bit Integer. Specify if the clip is full-range (True) or TV-range (False). fulls is for input, fulld for output.
video = haf.Weave(video, tff=True)
video = mvs.AssumeTFF(video)
video.set_output()
return True
main()

x64 vapoursynth, PC is an i3820 4-core, with 16Gb mem, Nvidia 750Ti with latest nvidia driver.
Most of the memory appears to be free during the encode at least for the first 10 mins, 8 hyperthrerads were running at about 20% cpu each.
Other non-Vine encodes run ok.

[feisty2]

try Myrsloik's build of fmtconv, https://dl.dropboxusercontent.com/u/73468194/fmtconvwithdebugsymbols.7z
it ain't crashed since the day I got it....
cretindesalpes' build always crashes randomly like probably at any moment for some mysterious reason(s)...

[Myrsloik]

try Myrsloik's build of fmtconv, https://dl.dropboxusercontent.com/u/73468194/fmtconvwithdebugsymbols.7z
it ain't crashed since the day I got it....
cretindesalpes' build always crashes randomly like probably at any moment for some mysterious reason(s)...

You could also try your luck with the internal resizer, that should work just as well for this. But if you do find a crash try to figure out where it happens so it can be fied.

[hydra3333]

You could also try your luck with the internal resizer, that should work just as well for this.
If I interpret that correctly ...
http://www.vapoursynth.com/doc/functions/resize.html
OK, not sure how to use the resizer to convert to 32bit float and then back to 8bit ... advice welcome.

try Myrsloik's build of fmtconv, https://dl.dropboxusercontent.com/u/73468194/fmtconvwithdebugsymbols.7z
it ain't crashed since the day I got it....
Thank you. Although, I admit to preferring not using software from dropboxes ... how quaint of me.

[jackoneill]

To convert to 32 bit float and back:

original_format = video.format

float_format = core.register_format(original_format.color_family, vs.FLOAT, 32, original_format.subsampling_w, original_format.subsampling_h)

video = core.resize.Bicubic(video, format=float_format.id)

...

video = core.resize.Bicubic(video, format=original_format.id)

[TheFluff]

I gotta say it's pretty cool that the resizer actually supports resizing to a format programmatically registered at runtime

I mean, it's not a scenario that's bound to come up often but when it does it's pretty awesome to have that functionality

[hydra3333]

Yes cool.
core = vs.get_core(accept_lowercase=True) # leave off threads=8 so it auto-detects threads
def main():
video = core.d2v.Source(r'T:\HDTV\WDTVlive\MP4-VS\x64-1\test.d2v')
video = core.std.Trim(video,first=0,length=1000)
video = core.std.SeparateFields(video, tff=True)
original_format = video.format # http://forum.doom9.org/showthread.php?p=1780587#post1780587
float_format = core.register_format(original_format.color_family, vs.FLOAT, 32, original_format.subsampling_w, original_format.subsampling_h) # http://forum.doom9.org/showthread.php?p=1780587#post1780587
video = core.resize.Bicubic(video, format=float_format.id) # http://forum.doom9.org/showthread.php?p=1780587#post1780587
#video = Vine.Dehalo(video, radius=[2, None])
video = video = core.resize.Bicubic(video, format=original_format.id) # http://forum.doom9.org/showthread.php?p=1780587#post1780587
video = haf.Weave(video, tff=True)
video = mvs.AssumeTFF(video)
video.set_output()
return True
main()
Gives me only 49.5 fps x264 encoding speed, without any other processing like deblocking and qtgmc and sharpening etc :) The "cost of doing business" I suppose.

On the other matter of the previsouly reported crash, [hang on, ignore this while I re-try] if I un-comment the Vine.Dehalo line then vspipe crashes with th Windows dialogue popup box and no python trace, which suggests that maybe Vine.Dehalo could be a tad less than stable from time to time.[/hang on, ignore this while I re-try]
Windows event log says error event 1000 :-
Faulting application name: VSPipe.exe, version: 0.0.0.0, time stamp: 0x57bd7828
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x0000000000000000
Faulting process ID: 0xb64
Faulting application start time: 0x01d20e7b2fbaa584
Faulting application path: C:\SOFTWARE\Vapoursynth-x64\VSPipe.exe
Faulting module path: unknown
Report ID: a8707734-60ee-4dd5-8b45-35ed7814a9e2
Faulting package full name:
Faulting package-relative application ID:
Scrub that, I can't reproduce the issue. edit: yes I can, see over at KNLmeansCL http://forum.doom9.org/showthread.php?p=1780660#post1780660
The only thing I can say is that other scripts are very reliable, but not this one containing these conversions and vine. When I add the vine line it sits there and takes about 3 to 5 minutes before actual x264 encoding commences by displaying frame number and fps etc ... then it goes at about 0.35 fps.

[feisty2]

Vine is stable, cretindesalpes' build of fmtconv isn't so
And main() is redundant, Python is a scripting language

[hydra3333]

OK agreed, thank you. It just runs a bit glacially for me even with the internal resizers changing the format back and forth. I wonder if it's related in any way to use of KNLmeansCL for me.

The main() approach is used so I can position that statement last in the script, and declare a number of home-grown functions just underneath the "main code at the top".
Python apparently doesn't seem to like calling functions which are declared later in the script.

[Elegant]

Haven't had the chance to setup my encoding box again so I thought I'd ask: Is this appropriate for anime/cartoons? Overkill? I have a few series on DVD that have poor transfers and suffer from halos. Thanks!

[feisty2]

Haven't had the chance to setup my encoding box again so I thought I'd ask: Is this appropriate for anime/cartoons? Overkill? I have a few series on DVD that have poor transfers and suffer from halos. Thanks!

it should work on cartoon stuff also I guess, but probably not worth it, Vine is pretty slow, and other lower quality but levels faster dehalo filters might be good enough for cartoons since getting cartoonized is one of the last things to worry about on literally cartoon vids

[brucethemoose]

it should work on cartoon stuff also I guess, but probably not worth it, Vine is pretty slow, and other lower quality but levels faster dehalo filters might be good enough for cartoons since getting cartoonized is one of the last things to worry about on literally cartoon vids

Some toons have more detailed, painted backgrounds that you have to worry about, but that's still true.

In fact, my problem is that Vine isn't strong enough... I've tried tweaking parameters, but it barely puts a dent in big sharpening halos like this:

http://i.cubeupload.com/YAZVwX.png

[Mystery Keeper]

Some toons have more detailed, painted backgrounds that you have to worry about, but that's still true.

In fact, my problem is that Vine isn't strong enough... I've tried tweaking parameters, but it barely puts a dent in big sharpening halos like this:

http://i.cubeupload.com/YAZVwX.png

You need to learn that miracles don't happen. And be skeptic about halo removing in general. AFAIR, there has been no method to remove them that wasn't destructive to other parts of video. What you've got is a horrible multi-peaks ringing. No hope to fight it. Better to not cause it.

[Nevilne]

"you can't expect this halo remover to remove these halos"

[brucethemoose]

No hope to fight it. Better to not cause it.

Unfortunately, that's the raw NTSC DVD rip. Yeah, it's that bad :(.

And yeah, I know... but there is a compromise somewhere between the Vine settings I've tried and DeHalo_Alpha, I just haven't found it yet.

[Mystery Keeper]

"you can't expect this halo remover to remove these halos"These multi-peaks? No, you can't. Narrow one peak halos - sure, you can. But expect it to screw up something else.

[feisty2]

Have you tried to raise "h" and "radius" up like a bit insane, especially "h", like "h=24"

[brucethemoose]

Have you tried to raise "h" and "radius" up like a bit insane, especially "h", like "h=24"

Not yet, thanks.


But holy moly... now I'm running this:

Vine.Dehalo(clip, radius=[8, 4], a=32, h=24, sharp=1.0, sigma=0.6, alpha=0.36, beta=32, cutoff=4, show=False)

And its taking like 8 seconds per 720x480 frame to encode. I'll get back to you when I get results.

[brucethemoose]

Have you tried to raise "h" and "radius" up like a bit insane, especially "h", like "h=24"

24 is way too strong, this is 20 with radius 4,2 (with a straight spline36 scale for reference):

http://www.screenshotcomparison.com/comparison/185123

Look at the guy's eyes, in particular.

I'm messing with it now, and 12-16 is looking better... the higher order halos aren't completely gone, but I might be able to mask them with another filter now.


This is some great code though, thanks feisty2.

[feisty2]

Now I think a large "h" is not it, something like "h=8" might be enough already
You need to disable all mask stuff in Vine cuz the halos are roaming VERY far from the exact edges, pointless tryna "mask" them out

[feisty2]

I'll add such option(disable mask stuff) after college

[brucethemoose]

Now I think a large "h" is not it, something like "h=8" might be enough already
You need to disable all mask stuff in Vine cuz the halos are roaming VERY far from the exact edges, pointless tryna "mask" them out

Sounds good.

However, if I'm looking at it right, I think the mask is actually reaching WAY out to the halos as it should.

http://u.cubeupload.com/brucethemoose/Disc1Alphavpymask.png

I don't think disabling it would help. There are other scenes with much more intricate painted backgrounds, and it looks like the mask will help preserve those.

EDIT: I could be wrong though, maybe I'm misunderstanding what the mask does...

So for now, what parameters should I adjust for the mask?

[feisty2]

radius=[10, None], sigma=1.6, alpha=0.48 works on your first screenshot
I'm on cellphone right now so...

[brucethemoose]

radius=[10, None], sigma=1.6, alpha=0.48 works on your first screenshot
I'm on cellphone right now so...

Sorry for not responding earlier, I went down the deinterlacing/IVTC rabbit hole.

What h value did you use?

[feisty2]

r5:
new parameter "masking"

[feisty2]

@brucethemoose

clp = Vine.Dehalo(clp, h=6, masking=False, cutoff=1, sharp=0.4)
clp = Vine.Dehalo(clp, h=4, masking=False, cutoff=1, sharp=0.6)
clp = Vine.Dehalo(clp, h=2, masking=False, cutoff=1, sharp=0.8)

http://i.imgur.com/VaQQ8Ud.png

[feisty2]

and of course you can go for a custom mask for this specific vid and protect soft features a bit more

[brucethemoose]

Fantastic, thanks! I'm playing with this now, might upload some before/after screenshots/clips later to show Vine off, as those Avatar DVDs are a DeHalo torture test.

[brucethemoose]

@brucethemoose

clp = Vine.Dehalo(clp, h=6, masking=False, cutoff=1, sharp=0.4)
clp = Vine.Dehalo(clp, h=4, masking=False, cutoff=1, sharp=0.6)
clp = Vine.Dehalo(clp, h=2, masking=False, cutoff=1, sharp=0.8)



Out of curiosity, why call Dehalo 3 times instead of just increasing h? Does it help with higher order halos or something?

[brucethemoose]

In certain material, Vine has trouble correctly detecting the actual edges:

Look at the edges of the cards here:

http://www.screenshotcomparison.com/comparison/185510

In that crack between the 2 black lines, it's so bad that the halos themselves are detected as edges, and it ends up blurring the actual edges. The mask and strength settings don't seem to affect that.



Feisty2, I know this is a live-action filter, but could you add an option that gives priority to black lines during edge detection? This would help oversharpened animation dramatically.

[feisty2]

there's no edge detection with "masking=False"
you do a halo/edge mask for your specific video on your own

[Tormaid]

there's no edge detection with "masking=False"
you do a halo/edge mask for your specific video on your own
^This

Only sane way to do it.

[brucethemoose]

^This

Only sane way to do it.

Well it's not a mask problem really. That's an area I definitely need to dehalo (and debug the mask is showing it), but the dehalo process itself isn't working.


I can't automatically filter it out and just live with the halos either. It's an edge just like everything else the mask sees, AFAIK there's no way to get a mask to detect edges, but NOT detect edges where there's a white halo in the inner perimeter of an object with a black outline.

EDIT: MadVR's anti ringing seems to have the same idea:

https://i.imgur.com/LtIULrD.png

It has a seperate setting for working with light edges.

[feisty2]

@brucethemoose

clp0 = clp
mask = Vine.Dehalo(clp,sigma=0.001,radius=[0,0],alpha=0.76,h=12,show=True)
for i in range(6):
mask = core.std.Maximum(mask)
clp = Vine.Dehalo(clp,h=4.8,masking=False)
clp = Vine.Dehalo(clp,h=2,masking=False,cutoff=1)
clp = Vine.Dehalo(clp,h=2,masking=False,cutoff=1)
clp = Vine.Dehalo(clp,h=2,masking=False,cutoff=1)
clp = core.std.MaskedMerge(clp0,clp,mask)

http://i.imgur.com/Xk8l8d1.png

[feisty2]

I don't really get "edge detection" and whatever other stuff you were talking about
but I got my own life so please stop expecting to be spoon fed

[brucethemoose]

I know, I wasn't asking for an update after your other reply, just speculating. I don't fully understand how DeHalo itself works, which means I should look at the code itself.

That looks much better, thanks. The range mask is a good idea.

[JackCY]

readme and more
https://github.com/IFeelBloated/Vine

Vine does not turn your video to a piece of cartoonized shit like other dehalo filters do.
it's slow but the performance is kind of practical still.

I've tried this script and... well it works, kind of... the parameters have to be pushed out the wazoo so thick halos are detected, such as example given on GitHub. But, the performance is awful, outright awful. Even with KNLMeans set to a=1 via Dehalo(a=1). Even with KNLMeans commented out in the Vine.py code so there should be no other crazy expensive operation... well... there is and I don't have another day to find which it is but it certainly is CPU limited not GPU limited in OpenCL.

No matter what Vine runs < 1fps on 1920x816px clip. Around 0.60fps, maybe.

Do I have some bad conversion copied from this thread to get that format Vine wants or is Vine so slow it's practically impractical? < 1fps certainly isn't practical, I've expected something like 10ms delay from processing per full HD frame not 1600ms.

At this speed the script is only good for processing a few still frames, that's about it.

What speeds is OP/creator getting when processing full HD?
From what I see the RAM keeps going +3GB what I would say is per frame, so each frame RAM consumption goes up by 3GB and then drops back down. WTH is eating 3GB.

The moment I comment out Vine out of my script... 80fps. Hell I can do 300fps, doesn't help anyway.

If this script is running 1000ms per frame, you should really put it in the OP or readme of the script as this is not usable for videos one bit in current state of the script and hardware available.

Are there any other dehalo VapourSynth options? Ones that do run fast for video processing? Don't care what visual quality as long as it's fast and usable.

[Mystery Keeper]

If you don't care about visual quality - you should just go with the halos.

[JackCY]

If you don't care about visual quality - you should just go with the halos.

I do but not to the point of 1+s processing per frame. MadVR has no problem detecting halos at very fast speed, any similar algorithm in VapourSynth that can be used to process videos before encoding and adjust the strength?

[feisty2]

I do but not to the point of 1+s processing per frame. MadVR has no problem detecting halos at very fast speed, any similar algorithm in VapourSynth that can be used to process videos before encoding and adjust the strength?

Madvr works on gpu
nnedi3 works on cpu

[JackCY]

Madvr works on gpu
nnedi3 works on cpu
nnedi3 shouldn't run on CPU, none image processing plugins should. It's insane to run that alg. on CPU.

Output 2000 frames in 88.44 seconds (22.62 fps)
# for i in range(2):
# clean = Transpose(NNEDI(Transpose(NNEDI(clean, **nnedi_args)), **nnedi_args))
# clean = Resample(clean, src.width, src.height, sx=-1.25, sy=-1.25, kernel="cubic", a1=-sharp, a2=0)
vs
Output 20 frames in 47.16 seconds (0.42 fps)
for i in range(2):
clean = Transpose(NNEDI(Transpose(NNEDI(clean, **nnedi_args)), **nnedi_args))
clean = Resample(clean, src.width, src.height, sx=-1.25, sy=-1.25, kernel="cubic", a1=-sharp, a2=0)

Difference? Usable vs unusable, noisier edges (close to original, only slightly cleaned) vs cleaner edges (even more cleaner than original). The edges might be changed by doing some other cheaper operation, such as convolution blur to reduce sharpness or the opposite since the sharpness argument is now defunct, no problem to emulate 0.5 sharpness which helped to remove more of the halos.

Adding convolution blur changed the performance to...
Output 2000 frames in 89.86 seconds (22.26 fps)
Probably a margin of error. But say -0.50fps, 0.984x.

Commenting out 3 lines = 53.85x faster.
This is with "a=3" which is enough for KNLMeansCL. Radius 3.

The next one to go seems to be fmtc:
Resample = core.fmtc.resample
MakeDiff = core.std.MakeDiff
MergeDiff = core.std.MergeDiff
def inline(src):
upsmp = Resample(src, src.width * 2, src.height * 2, kernel="gauss", a1=100, **fmtc_args)
clip = Resample(upsmp, src.width, src.height, kernel="gauss", a1=p, **fmtc_args)
return clip
If there is a VS native replacement that's faster.

original cutoff
Output 2000 frames in 90.55 seconds (22.09 fps)
Output 1000 frames in 46.26 seconds (21.62 fps)
convolution 5x5
Output 2000 frames in 85.34 seconds (23.43 fps)
Output 1000 frames in 43.60 seconds (22.93 fps)
convolution 3x3
Output 2000 frames in 74.06 seconds (27.01 fps)
Output 1000 frames in 37.46 seconds (26.69 fps)

Convolution 5x5 not really worth it, looks similar too. 3x3 limits the dehalo ability but it's faster by 5fps = 1.23x, not bad at all if one doesn't mind the reduced halo reduction.

Canny, no difference.
Can't get rid of the necessary format conversion, dunno why format changes somewhere and then causes an error, what operation is changing the format in Vine?
This is what was failing without converting to 32bit float:
clean = helpers.cutoff(src, clean, cutoff)
dif = MakeDiff(src, clean)
Right between the nlmeans in dehalo.

---

In the end this script can run not 0.4fps but 27fps, 67x faster and usable, though still slow from my POV.

[ChaosKing]

Are there any other dehalo VapourSynth options? Ones that do run fast for video processing? Don't care what visual quality as long as it's fast and usable.

DeHalo_alpha & YAHR:
http://forum.doom9.org/showthread.php?t=166582

[captaiŋadamo]

nnedi3 shouldn't run on CPU, none image processing plugins should. It's insane to run that alg. on CPU.

So you're volunteering to rewrite all Avisynth/Vapoursynth plugins to run on the GPU? :sly:

[JackCY]

DeHalo_alpha & YAHR:
http://forum.doom9.org/showthread.php?t=166582

Thanks will try them if I can hunt down all the dependencies.

Result:
DeHalo_alpha
# rx=3., ss = 1.0 # Output 1000 frames in 9.56 seconds (104.62 fps) # disables resampling and uses repair instead
# rx=3., ss = 1.5 # Output 1000 frames in 14.41 seconds (69.38 fps)
YAHR
# blur=2, depth=32 # Output 1000 frames in 33.02 seconds (30.28 fps)
Vine.Dehalo (no nnedi3, cutoff original)
# [3, None], a=3, sigma=1.3, alpha=0.27, h=24, cutoff=4 # Output 1000 frames in 46.09 seconds (21.70 fps)

DeHalo_alpha - general dehalo, easily strong, relatively easy to tweak it's strength up and down
YAHR - only affect 1st halo ring not 2nd, such as black + white halos only white gets removed, black stays, for me impossible to make stronger or weaker I suppose too
Vine.Dehalo - nnedi3 needs to go for it to be usable, might use the DeHalo_alpha resampling and repair instead but simply none is fine too and keeps edges original instead of smoothed via nnedi3 resampling

I even doubt any resamplings in these scripts are done with a mask, so that only used pixels (filtered edges) are resampled say 5% of the image and not all pixels 100% of the image, which would alone provide a big speed up. Resampler scripts probably weren't made with that in mind.

On my test sample with denoise etc.:
DeHalo_alpha
Output 1000 frames in 16.35 seconds (61.16 fps)
encoded 1000 frames, 68.57 fps, 4505.42 kbps, 22.40 MB
encode time 0:00:14 / CPU Usage: 8.68%

No dehalo
Output 1000 frames in 14.83 seconds (67.42 fps)
encoded 1000 frames, 80.33 fps, 5468.65 kbps, 27.19 MB
encode time 0:00:12 / CPU Usage: 11.23%

[feisty2]

I even doubt any resamplings in these scripts are done with a mask, so that only used pixels (filtered edges) are resampled say 5% of the image and not all pixels 100% of the image, which would alone provide a big speed up. Resampler scripts probably weren't made with that in mind.

"pscrn" is the internal masking for nnedi3, set it 0 and you'll know how slow nnedi3 could be when its working on the entire image..

The edges might be changed by doing some other cheaper operation, such as convolution blur to reduce sharpness or the opposite since the sharpness argument is now defunct, no problem to emulate 0.5 sharpness which helped to remove more of the halos.


This is with "a=3" which is enough for KNLMeansCL. Radius 3.

supersampling(nnedi3) is supposed to remove aliasing, there's gonna be the aliasing-ringing tradeoff like all convolution filters, supersampling makes sure you get the best result from both sides and simply avoid the tradeoff.

a=3 is not "non-local"

Vine is a quality solution rather than realtime solution, but go ahead and remove things you think are pointless, I'm not stopping you...

[Motenai Yoda]

"pscrn" is the internal masking for nnedi3, set it 0 and you'll know how slow nnedi3 could be when its working on the entire image...
actually nnedi3 pscrn mask where nnedi is applied but all the other pixels are rescaled with a bicubic resample, so a resample is still applied to the entire frame

[feisty2]

actually nnedi3 pscrn mask where nnedi is applied but all the other pixels are rescaled with a bicubic resample, so a resample is still applied to the entire frame

Bicubic is negligible performance-wise

[xekon]

I am trying to use Vine, as soon as I hit F5 to generate a preview in VSedit, it crashes. After reading though this thread I am assuming its because of one of my prerequisites. I seen the post about using Myrsloik's build of fmtconv. Where could I find his current build? I am currently using R20 of cretindesalpes' build. I am not sure where to look for a crash log. I dont know if Vapoursynth or VSedit has one when its a Fatal Crash.

here is my code:
original_format = ret.format
float_format = core.register_format(original_format.color_family, vs.FLOAT, 32, original_format.subsampling_w, original_format.subsampling_h)
ret = core.resize.Bicubic(ret, format=float_format.id)
ret = Vine.Dehalo(ret, [3, None], sigma=1.6, alpha=0.18, h=12, sharp=0.5, cutoff=2)
ret = Vine.Dehalo(ret, [3, None], sigma=2.2, alpha=0.18, h=24, sharp=0.5, cutoff=2)
ret = core.resize.Bicubic(ret, format=original_format.id)
ret.set_output()

EDIT: I found and tried jackoneill's fmtconv: https://github.com/dubhater/fmtconv/tree/master/build/unix
Vine still crashes immediately every time I hit F5, not sure how to go about finding the culprit.

[feisty2]

r5:
a blending operation is now appended after super-sampling, weight determined by the "sharp" parameter
mathematical expression: weight = constant * sharp * ln(1 + 1 / (constant * sharp)) (constant determines how fast weight gets near 1.0 as sharp gets larger)
the mathematical limit for weight is 0 (simply returns the resampled result) as sharp goes infinitely close to 0, or 1 (simply returns the clip before resampling) as sharp goes towards infinity
now if you feel like to barf any time math is somewhere around you, and would like not to analyze that weight expression with differential calculus, all you gotta know is a larger "sharp" value also results in less percentage of super-sampling

[bebs]

It seems that KNLMeansCL breaks Vine, i keep having an error KNLMeansCL: 'rclip' does not match the source clip! when using KNLMeansCL 1.0.0.
KNLMeansCL .77 is working fine.

[Jindadil007]

It seems that KNLMeansCL breaks Vine, i keep having an error KNLMeansCL: 'rclip' does not match the source clip! when using KNLMeansCL 1.0.0.
KNLMeansCL .77 is working fine.

Correctly Reported...i am also using .77 for the same reason...

[feisty2]

It seems that KNLMeansCL breaks Vine, i keep having an error KNLMeansCL: 'rclip' does not match the source clip! when using KNLMeansCL 1.0.0.
KNLMeansCL .77 is working fine.

v1.0.0 is buggy, I already reported that bug in the KNLMeansCL thread

[feisty2]

r6:
redesigned the super-sampling process

[feisty2]

r7:
redesigned the dehalo filter (Basic+Final)

[Jindadil007]

r7:
redesigned the dehalo filter (Basic+Final)

Update on New Script : Dehalo command is not recognised anymore. Tested with Basic + Final ... Earlier (Script) fps was 0.23 now it has dropped to 0.14 ... Plus there are Memory Leaks with same dependencies :

Core freed but 87 filter instances still exist
Core freed but 87 filter instances still exist
Core freed but 1525568512 bytes still allocated in framebuffers
Core freed but 1525568512 bytes still allocated in framebuffers

[feisty2]

Update on New Script : Dehalo command is not recognised anymore. Tested with Basic + Final ... Earlier (Script) fps was 0.23 now it has dropped to 0.14 ... Plus there are Memory Leaks with same dependencies :

Core freed but 87 filter instances still exist
Core freed but 87 filter instances still exist
Core freed but 1525568512 bytes still allocated in framebuffers
Core freed but 1525568512 bytes still allocated in framebuffers

please don't use the latest version for now, I'll let you know when it's ready

[Jindadil007]

please don't use the latest version for now, I'll let you know when it's ready

Oops !!! I replaced the earlier script file and I can't find it anymore. Pl. share from where i can download the previous script...Thanks

[feisty2]

Oops !!! I replaced the earlier script file and I can't find it anymore. Pl. share from where i can download the previous script...Thanks

https://github.com/IFeelBloated/Vine/blob/2ca260dc287f59ef124375e209d86a7b3cda1150/Vine.py

[Jindadil007]

https://github.com/IFeelBloated/Vine/blob/2ca260dc287f59ef124375e209d86a7b3cda1150/Vine.py

:thanks:

[feisty2]

:thanks:

you can use the latest version now if you want to
and..
1. stop complaining about memory leaks in this thread, if there're memory leaks in certain plugin(s), Vine is not what's causing all that, as I already told you here, https://forum.doom9.org/showpost.php?p=1795833&postcount=37
complain in the thread for the exact plugin that has memory leaks if you know which plugin is the one to blame, or
complain in the main vaporsynth thread if you don't, Myrsloik or jackoneill will probably help you out if they're in a good mood..
2. yeah, the new version is slower but the quality is better, you can stick to the previous version if you're not okay with the performance.

[Jindadil007]

you can use the latest version now if you want to
and..
1. stop complaining about memory leaks in this thread, if there're memory leaks in certain plugin(s), Vine is not what's causing all that, as I already told you here, https://forum.doom9.org/showpost.php?p=1795833&postcount=37
complain in the thread for the exact plugin that has memory leaks if you know which plugin is the one to blame, or
complain in the main vaporsynth thread if you don't, Myrsloik or jackoneill will probably help you out if they're in a good mood..
2. yeah, the new version is slower but the quality is better, you can stick to the previous version if you're not okay with the performance.
Thanks...Would Check it surely...

[Cary Knoop]

Running Vine using a SMPTE RP 219 test chart with the following commands:


ref = Vine.Basic(c, h=31.18, sharp=0.48)
ref = Vine.Basic(ref, h=13.86, sharp=0.16)
c = Vine.Final([c, ref], [Vine.Super(c), Vine.Super(ref)], [6, 2, 4], sigma=2.2, alpha=0.18)


I notice a levels adjustment. Source is video levels, after Vine the levels are squeezed.

[poisondeathray]

Running Vine using a SMPTE RP 219 test chart with the following commands:


ref = Vine.Basic(c, h=31.18, sharp=0.48)
ref = Vine.Basic(ref, h=13.86, sharp=0.16)
c = Vine.Final([c, ref], [Vine.Super(c), Vine.Super(ref)], [6, 2, 4], sigma=2.2, alpha=0.18)


I notice a levels adjustment. Source is video levels, after Vine the levels are squeezed.

How "squeezed ?" What is your full script ?

I noticed no significant change in levels. Slightly different source. YUV422P8 bars with pluge

pre / post vine
https://s9.postimg.org/m4y9dncsf/input_postvine.png (https://postimages.org/)

[Cary Knoop]

Squeezed as if the video levels are interpreted as full range and "squeezed" to video.


source = 'VideoLevelsRec709CineformSMPTE1080P.mov'
c = core.lsmas.LWLibavSource(source)
c = core.fmtc.bitdepth (c, bits=32)
ref = Vine.Basic(c, h=31.18, sharp=0.48)
ref = Vine.Basic(ref, h=13.86, sharp=0.16)
c = Vine.Final([c, ref], [Vine.Super(c), Vine.Super(ref)], [6, 2, 4], sigma=2.2, alpha=0.18)
return core.fmtc.bitdepth (c, bits=8)
c.set_output()


Was your source full range?

[poisondeathray]

Was your source full range?

No, it's legal range video levels except for the pluge; you can see the waveform in IRE in my post, I added it

Check each step with histogram ; for example comment out everything, except for lsmash source filter, check levels; the vine lines, check levels, comment out fmtc lines, check levels etc...

Vine didn't do anything significant to levels in my test (beyond the expected slight changes in edges)

[Cary Knoop]

Tested this again, and well, I can't reproduce the situation.
I think we have to call it a user error (that's me).
I am sorry.

By the way, Vine works great!

I was wondering how much change it would take to restrict the adjustments to horizontal adjustments only (i.e. reducing vertical stripes while keeping horizontal stripes alone)?

[zerowalker]

How do you use the Dehalo filter?
I can't make it out:(

[Cary Knoop]

Vine works great but the fmtc_args dictionary settings change the levels for float32 processing, emptying the fmtc_args dictionary solves the problem for me.

[WaxCyl]

Does anyone know if there is an update for Vine to work with the latest version of Vapoursynth? I have been unable to find one.
I have had an attempt to do this myself, but with no luck. Def NLmeans brings an error message.
Here is my version of Vine so far:

[WaxCyl]

import vapoursynth as vs
import math

fmtc_args = dict(fulls=True, fulld=True)
msuper_args = dict(hpad=0, vpad=0, sharp=2, levels=0, chroma=False)
manalyze_args = dict(search=3, truemotion=False, trymany=True, levels=0, badrange=-24, divide=0, dct=0, chroma=False)
mrecalculate_args = dict(truemotion=False, search=3, smooth=1, divide=0, dct=0, chroma=False)
mdegrain_args = dict(plane=0, thscd1=16711680.0, thscd2=255.0)
canny_args = dict(mode=1, op=0)
nnedi_args = dict(field=1, dh=True, nns=4, qual=2, etype=1, nsize=0)

class get_core:
def __init__(self):
self.MSuper = self.core.mvsf.Super
self.MAnalyze = self.core.mvsf.Analyze
self.MRecalculate = self.core.mvsf.Recalculate
self.MDegrain = self.core.mvsf.Degrain
self.KNLMeansCL = self.core.haf.KNLMeansCL
self.Canny = self.core.tcanny.TCanny
self.NNEDI = self.core.nnedi3.nnedi3
self.RGB2OPP = self.core.bm3d.RGB2OPP
self.OPP2RGB = self.core.bm3d.OPP2RGB
self.Resample = self.core.fmtc.resample
self.Maximum = self.core.std.Maximum
self.Minimum = self.core.std.Minimum
self.Expr = self.core.std.Expr
self.Merge = self.core.std.Merge
self.MakeDiff = self.core.std.MakeDiff
self.MergeDiff = self.core.std.MergeDiff
self.Crop = self.core.std.CropRel
self.AddBorders = self.core.std.AddBorders
self.Transpose = self.core.std.Transpose
self.Inflate = self.core.std.Inflate
self.MaskedMerge = self.core.std.MaskedMerge
self.ShufflePlanes = self.core.std.ShufflePlanes
self.SetFieldBased = self.core.std.SetFieldBased

def CutOff(self, low, hi, p):
def inline(src):
upsmp = self.Resample(src, src.width*2, src.height*2, kernel="gauss", a1=100, **fmtc_args)
clip = self.Resample(upsmp, src.width, src.height, kernel="gauss", a1=p, **fmtc_args)
return clip
hif = self.MakeDiff(hi, inline(hi))
clip = self.MergeDiff(inline(low), hif)
return clip

def Pad(self, src, left, right, top, bottom):
w = src.width
h = src.height
clip = self.Resample(src, w+left+right, h+top+bottom, -left, -top, w+left+right, h+top+bottom, kernel="point", **fmtc_args)
return clip

def NLMeans(self, src, d, a, s, h, rclip):
pad = self.AddBorders(src, a+s, a+s, a+s, a+s)
rclip = self.AddBorders(rclip, a+s, a+s, a+s, a+s) if rclip is not None else None
nlm = self.KNLMeansCL(pad, d=0, a=a, s=s, h=h, rclip=rclip)
clip = self.Crop(nlm, a+s, a+s, a+s, a+s)
return clip

def XYClosest(self, src1, src2, ref):
clip = self.Expr([src1, src2, ref], "x z - abs y z - abs > y x ?")
return clip

class internal:
def dilation(core, src, radius):
for i in range(radius):
src = core.Maximum(src)
return src

def erosion(core, src, radius):
for i in range(radius):
src = core.Minimum(src)
return src

def closing(core, src, radius):
clip = internal.dilation(core, src, radius)
clip = internal.erosion(core, clip, radius)
return clip

def opening(core, src, radius):
clip = internal.erosion(core, src, radius)
clip = internal.dilation(core, clip, radius)
return clip

def gradient(core, src, radius):
erosion = internal.erosion(core, src, radius)
dilation = internal.dilation(core, src, radius)
clip = core.Expr([dilation, erosion], "x y -")
return clip

def tophat(core, src, radius):
opening = internal.opening(core, src, radius)
clip = core.Expr([src, opening], "x y -")
return clip

def blackhat(core, src, radius):
closing = internal.closing(core, src, radius)
clip = core.Expr([src, closing], "y x -")
return clip

def super(core, src, pel):
src = core.Pad(src, 128, 128, 128, 128)
clip = core.std.transpose(core.nnedi3.nnedi3(core.std.transpose(core.nnedi3.nnedi3(src, **nnedi_args)), **nnedi_args))
if pel == 4:
clip = core.std.transpose(core.nnedi3.nnedi3(core.std.transpose(core.nnedi3.nnedi3(clip, **nnedi_args)), **nnedi_args))
return clip

def basic(core, src, a, h, sharp, cutoff):
c1 = 0.3926327792690057290863679493724
c2 = 18.880334973195822973214959957208
c3 = 0.5862453661304626725671053478676
weight = c1 * sharp * math.log(1.0 + 1.0 / (c1 * sharp))
h_refine = c2 * math.pow(h / c2, c3)
upsampled = core.std.Transpose(core.nnedi3.nnedi3(core.std.Transpose(core.nnedi3.nnedi3(src, **nnedi_args)), **nnedi_args))
upsampled = core.NLmeans(upsampled, a, 0, h, None)
resampled = core.Resample(upsampled, src.width, src.height, sx=-0.5, sy=-0.5, kernel="cubic", a1=-sharp, a2=0)
clean = core.NLmeans(src, a, 0, h, None)
clean = core.Merge(resampled, clean, weight)
clean = core.CutOff(src, clean, cutoff)
dif = core.MakeDiff(src, clean)
dif = core.NLmeans(dif, a, 1, h_refine, clean)
clip = core.MergeDiff(clean, dif)
return clip

def final(core, src, super, radius, pel, sad, sigma, alpha, beta, masking, show):
constant = 0.0009948813682897925944723492342
me_sad = constant * math.pow(sad, 2.0) * math.log(1.0 + 1.0 / (constant * sad))
if masking:
mask = core.Canny(src[1], sigma=sigma, **canny_args)
mask = core.Expr(mask, "x {alpha} + {beta} pow {gamma} - 0.0 max 1.0 min".format(alpha=alpha, beta=beta, gamma=math.pow(alpha, beta)))
expanded = internal.dilation(core, mask, radius[1])
closed = internal.closing(core, mask, radius[1])
mask = core.Expr([expanded, closed, mask], "x y - z +")
for i in range(radius[2]):
mask = core.Inflate(mask)
if show:
return mask
for i in range(2):
src[i] = core.Pad(src[i], 128, 128, 128, 128)
src[1] = core.MakeDiff(src[1], src[0])
super[1] = core.MakeDiff(super[1], super[0]) if super[0] is not None and super[1] is not None else None
blankdif = core.Expr(src[0], "0.0")
supersearch = core.MSuper(src[0], pelclip=super[0], rfilter=4, pel=pel, **msuper_args)
superdif = core.MSuper(src[1], pelclip=super[1], rfilter=2, pel=pel, **msuper_args)
vmulti = core.MAnalyze(supersearch, radius=radius[0], overlap=64, blksize=128, **manalyze_args)
vmulti = core.MRecalculate(supersearch, vmulti, overlap=32, blksize=64, thsad=me_sad, **mrecalculate_args)
vmulti = core.MRecalculate(supersearch, vmulti, overlap=16, blksize=32, thsad=me_sad, **mrecalculate_args)
vmulti = core.MRecalculate(supersearch, vmulti, overlap=8, blksize=16, thsad=me_sad, **mrecalculate_args)
vmulti = core.MRecalculate(supersearch, vmulti, overlap=4, blksize=8, thsad=me_sad, **mrecalculate_args)
vmulti = core.MRecalculate(supersearch, vmulti, overlap=2, blksize=4, thsad=me_sad, **mrecalculate_args)
vmulti = core.MRecalculate(supersearch, vmulti, overlap=1, blksize=2, thsad=me_sad, **mrecalculate_args)
averaged_dif = core.MDegrain(src[1], superdif, vmulti, thsad=sad, **mdegrain_args)
averaged_dif = core.XYClosest(averaged_dif, src[1], blankdif)
averaged_dif = core.Crop(averaged_dif, 128, 128, 128, 128)
src[0] = core.Crop(src[0], 128, 128, 128, 128)
clean = core.MergeDiff(src[0], averaged_dif)
clip = core.MaskedMerge(src[0], clean, mask) if masking else clean
return clip

def Super(src, pel=4):
if not isinstance(src, vs.VideoNode):
raise TypeError("Vine.Super: src has to be a video clip!")
elif src.format.sample_type != vs.FLOAT or src.format.bits_per_sample < 32:
raise TypeError("Vine.Super: the sample type of src has to be single precision!")
if not isinstance(pel, int):
raise TypeError("Vine.Super: pel has to be an integer!")
elif pel != 2 and pel != 4:
raise RuntimeError("Vine.Super: pel has to be 2 or 4!")
core = core
src = core.SetFieldBased(src, 0)
colorspace = src.format.color_family
if colorspace == vs.RGB:
src = core.RGB2OPP(src, 1)
if colorspace != vs.GRAY:
src = core.ShufflePlanes(src, 0, vs.GRAY)
clip = internal.super(core, src, pel)
del core
return clip

def Basic(src, a=32, h=6.4, sharp=1.0, cutoff=4):
if not isinstance(src, vs.VideoNode):
raise TypeError("Vine.Basic: src has to be a video clip!")
elif src.format.sample_type != vs.FLOAT or src.format.bits_per_sample < 32:
raise TypeError("Vine.Basic: the sample type of src has to be single precision!")
if not isinstance(a, int):
raise TypeError("Vine.Basic: a has to be an integer!")
elif a < 1:
raise RuntimeError("Vine.Basic: a has to be greater than 0!")
if not isinstance(h, float) and not isinstance(h, int):
raise TypeError("Vine.Basic: h has to be a real number!")
elif h <= 0:
raise RuntimeError("Vine.Basic: h has to be greater than 0!")
if not isinstance(sharp, float) and not isinstance(sharp, int):
raise TypeError("Vine.Basic: sharp has to be a real number!")
elif sharp <= 0.0:
raise RuntimeError("Vine.Basic: sharp has to be greater than 0!")
if not isinstance(cutoff, int):
raise TypeError("Vine.Basic: cutoff has to be an integer!")
elif cutoff < 1 or cutoff > 100:
raise RuntimeError("Vine.Basic: cutoff must fall in (0, 100]!")
core =vs. core
src = core.std.SetFieldBased(src, 0)
colorspace = src.format.color_family
if colorspace == vs.RGB:
src = core.RGB2OPP(src, 1)
if colorspace != vs.GRAY:
src = core.std. ShufflePlanes(src, 0, vs.GRAY)
clip = internal.basic(core, src, a, h, sharp, cutoff)
del core
return clip

continued next post

[WaxCyl]

def Final(src, super=[None, None], radius=[6, 1, None], pel=4, sad=400.0, sigma=0.6, alpha=0.36, beta=32.0, masking=True, show=False):
if not isinstance(src, list):
raise TypeError("Vine.Final: src has to be an array!")
elif len(src) != 2:
raise RuntimeError("Vine.Final: src has to contain 2 elements exactly!")
elif not isinstance(src[0], vs.VideoNode) or not isinstance(src[1], vs.VideoNode):
raise TypeError("Vine.Final: elements in src must be video clips!")
elif src[0].format.sample_type != vs.FLOAT or src[0].format.bits_per_sample < 32:
raise TypeError("Vine.Final: the sample type of src[0] has to be single precision!")
elif src[1].format.id != vs.GRAYS:
raise RuntimeError("Vine.Final: corrupted basic estimation!")
if not isinstance(super, list):
raise TypeError("Vine.Final: super has to be an array!")
elif len(super) != 2:
raise RuntimeError("Vine.Final: super has to contain 2 elements exactly!")
for i in range(2):
if not isinstance(super[i], vs.VideoNode) and super[i] is not None:
raise TypeError("Vine.Final: elements in super must be video clips or None!")
elif super[i] is not None:
if super[i].format.id != vs.GRAYS:
raise RuntimeError("Vine.Final: corrupted super clips!")
if not isinstance(radius, list):
raise TypeError("Vine.Final: radius parameter has to be an array!")
elif len(radius) != 3:
raise RuntimeError("Vine.Final: radius parameter has to contain 3 elements exactly!")
for i in range(2):
if not isinstance(radius[i], int):
raise TypeError("Vine.Final: radius[" + str(i) + "] has to be an integer!")
if radius[0] <= 0:
raise RuntimeError("Vine.Final: radius[0] has to be greater than 0!")
if radius[1] < 0:
raise RuntimeError("Vine.Final: radius[1] has to be no less than 0!")
if not isinstance(radius[2], int) and radius[2] is not None:
raise TypeError("Vine.Final: radius[2] has to be an integer or None!")
elif radius[2] is not None:
if radius[2] < 0:
raise RuntimeError("Vine.Final: radius[2] has to be no less than 0!")
if not isinstance(pel, int):
raise TypeError("Vine.Final: pel has to be an integer!")
elif pel != 1 and pel != 2 and pel != 4:
raise RuntimeError("Vine.Final: pel has to be 1, 2 or 4!")
if not isinstance(sad, float) and not isinstance(sad, int):
raise TypeError("Vine.Final: sad has to be a real number!")
elif sad <= 0:
raise RuntimeError("Vine.Final: sad has to be greater than 0!")
if not isinstance(alpha, float) and not isinstance(alpha, int):
raise TypeError("Vine.Final: alpha has to be a real number!")
elif alpha < 0.0 or alpha > 1.0:
raise RuntimeError("Vine.Final: alpha must fall in [0.0, 1.0]!")
if not isinstance(beta, float) and not isinstance(beta, int):
raise TypeError("Vine.Final: beta has to be a real number!")
elif beta <= 1.0:
raise RuntimeError("Vine.Final: beta has to be greater than 1.0!")
if not isinstance(masking, bool):
raise TypeError("Vine.Final: masking has to be boolean!")
if not isinstance(show, bool):
raise TypeError("Vine.Final: show has to be boolean!")
if not masking and show:
raise RuntimeError("Vine.Final: masking has been disabled, set masking True to show the halo mask!")
radius[2] = math.ceil(radius[1] / 2) if radius[2] is None else radius[2]
core = core
for i in range(2):
src[i] = core.SetFieldBased(src[i], 0)
super[i] = core.SetFieldBased(super[i], 0) if super[i] is not None else None
colorspace = src[0].format.color_family
if colorspace == vs.RGB:
src[0] = core.RGB2OPP(src[0], 1)
if colorspace != vs.GRAY:
src_color = src[0]
src[0] = core.ShufflePlanes(src[0], 0, vs.GRAY)
clip = internal.final(core, src, super, radius, pel, sad, sigma, alpha, beta, masking, show)
if colorspace != vs.GRAY:
clip = core.ShufflePlanes([clip, src_color], [0, 1, 2], vs.YUV)
if colorspace == vs.RGB:
clip = core.OPP2RGB(clip, 1)
del core
return clip

def Dilation(src, radius=1):
if not isinstance(src, vs.VideoNode):
raise TypeError("Vine.Dilation: src has to be a video clip!")
if not isinstance(radius, int):
raise TypeError("Vine.Dilation: radius has to be an integer!")
elif radius < 1:
raise RuntimeError("Vine.Dilation: radius has to be greater than 0!")
core = core
src = core.SetFieldBased(src, 0)
clip = internal.dilation(core, src, radius)
del core
return clip

def Erosion(src, radius=1):
if not isinstance(src, vs.VideoNode):
raise TypeError("Vine.Erosion: src has to be a video clip!")
if not isinstance(radius, int):
raise TypeError("Vine.Erosion: radius has to be an integer!")
elif radius < 1:
raise RuntimeError("Vine.Erosion: radius has to be greater than 0!")
core = core
src = core.SetFieldBased(src, 0)
clip = internal.erosion(core, src, radius)
del core
return clip

def Closing(src, radius=1):
if not isinstance(src, vs.VideoNode):
raise TypeError("Vine.Closing: src has to be a video clip!")
if not isinstance(radius, int):
raise TypeError("Vine.Closing: radius has to be an integer!")
elif radius < 1:
raise RuntimeError("Vine.Closing: radius has to be greater than 0!")
core = core
src = core.SetFieldBased(src, 0)
clip = internal.closing(core, src, radius)
del core
return clip

def Opening(src, radius=1):
if not isinstance(src, vs.VideoNode):
raise TypeError("Vine.Opening: src has to be a video clip!")
if not isinstance(radius, int):
raise TypeError("Vine.Opening: radius has to be an integer!")
elif radius < 1:
raise RuntimeError("Vine.Opening: radius has to be greater than 0!")
core = core
src = core.SetFieldBased(src, 0)
clip = internal.opening(core, src, radius)
del core
return clip

def Gradient(src, radius=1):
if not isinstance(src, vs.VideoNode):
raise TypeError("Vine.Gradient: src has to be a video clip!")
if not isinstance(radius, int):
raise TypeError("Vine.Gradient: radius has to be an integer!")
elif radius < 1:
raise RuntimeError("Vine.Gradient: radius has to be greater than 0!")
core = core
src = core.SetFieldBased(src, 0)
clip = internal.gradient(core, src, radius)
del core
return clip

def TopHat(src, radius=1):
if not isinstance(src, vs.VideoNode):
raise TypeError("Vine.TopHat: src has to be a video clip!")
if not isinstance(radius, int):
raise TypeError("Vine.TopHat: radius has to be an integer!")
elif radius < 1:
raise RuntimeError("Vine.TopHat: radius has to be greater than 0!")
core = core
src = core.SetFieldBased(src, 0)
clip = internal.tophat(core, src, radius)
del core
return clip

def BlackHat(src, radius=1):
if not isinstance(src, vs.VideoNode):
raise TypeError("Vine.BlackHat: src has to be a video clip!")
if not isinstance(radius, int):
raise TypeError("Vine.BlackHat: radius has to be an integer!")
elif radius < 1:
raise RuntimeError("Vine.BlackHat: radius has to be greater than 0!")
core = core
src = core.SetFieldBased(src, 0)
clip = internal.blackhat(core, src, radius)
del core
return clip





See the error message next page:

[WaxCyl]

Here is the error message:
2023-05-28 22:03:32.061
Failed to evaluate the script:
Python exception: No attribute with the name NLmeans exists. Did you mistype a plugin namespace?

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 3115, in vapoursynth._vpy_evaluate
File "src\cython\vapoursynth.pyx", line 3116, in vapoursynth._vpy_evaluate
File "E:/VapourProject.vpy", line 30, in
ref = Vine.Basic(clip, h=31.18, sharp=0.48)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\site-packages\Vine.py", line 208, in Basic
clip = internal.basic(core, src, a, h, sharp, cutoff)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\site-packages\Vine.py", line 115, in basic
upsampled = core.NLmeans(upsampled, a, 0, h, None)
^^^^^^^^^^^^
File "src\cython\vapoursynth.pyx", line 2628, in vapoursynth._CoreProxy.__getattr__
File "src\cython\vapoursynth.pyx", line 2483, in vapoursynth.Core.__getattr__
AttributeError: No attribute with the name NLmeans exists. Did you mistype a plugin namespace?

I will be very appreciative of any assistance

[richardpl]

Am I only one to spot that those morphological filters in here are using extremely inefficient algorithm?

[WaxCyl]

I think it it may be because Feisty2 placed the highest possible priority on image quality, above all else?

[richardpl]

I get same quality/output if not much better quality with much faster algorithm. Feel free to keep wasting CPU cycles on crappy morphological implementations.

[WaxCyl]

OK thanks for your thoughts. I have found Vine to be extremely good and found others to be not very effective. Maybe I am not fine tuning them effectively? What other halo remover would you suggest?

[richardpl]

I havent looked at dehalo quality, but I just noticed part of code use slow algorithms.

[WaxCyl]

I have noticed that Dogway has updated Feisty2's Plum and Oyster . Hopefully Vine gets updated eventually! :)