Dogway's Filters Packs

[Zetti]

MeGUI seems to be the problem.

Becuase i moved your scripts out of the plugin folder and created my basic script with only the index file, and i have moved your scripts back.

Now i have tested my basic script and it's working with AVSMeter.

With your scripts in the plugin folder, i couldnt create the basic script with MeGUI.

So from now on, i need to import all the required plugins and scripts from another folder into my encoding script.

[Dogway]

Well thanks for testing, I checked and in "MeGUI/tools/avs" folder you find the avisynth dll files, you can try replacing those with v3.7.2 test3, just an idea.

[kedautinh12]

You can try L-SMASH in megui. And i use extools in megui normally

[Zetti]

Maybe i should check the Avisynth and AvisynthWrapper versions one more time.

And testing L-SMASH.

If that not getting it to work, a seperate folder could be the solotion.

[kedautinh12]

If you use latest avs+ ver, you need change new AvisynthWrapper to use latest avs+ ver
https://forum.doom9.org/showthread.php?t=181351

[Zetti]

I got it working without any errors after replacing the old versions of Avisynth and AvisynthWrapper with new ones in MeGUI.

Can you both have a great and wonderfull day.

[Boulder]

There seems to be some sort of a memory leak in QTGMC, at least when SRestore is involved. This script causes the memory usage to creep up until it hits the set memory limit, the Avisynth basically stalls. It does proceed, but at a very, very slow pace.
Replacing QTGMC with real.finder's STGMC fixes the issue, the memory usage stays below 1GB all the time.

SetMemoryMax(8192)
DGSource("source.dgi") # 1080i, 29.97 fps
QTGMC(preset="medium", noisepreset="fast", tr2=0, sigma=0.25)
Spline36Resize(1280,720)
SRestore()

[Dogway]

I'm pretty sure it's a memory leak in pixel addressing, it has been reported almost 2 months ago, and recalled 2 weeks ago but pinterf has been turning a blind eye on it.

I will try to cobble up a version without pixel addressing so you can confirm. It's either that or a LUT bug.

[real.finder]

Originally Posted by Dogway

I'm pretty sure it's a memory leak in pixel addressing, it has been reported almost 2 months ago, and recalled 2 weeks ago but pinterf has been turning a blind eye on it.

I will try to cobble up a version without pixel addressing so you can confirm. It's either that or a LUT bug.

maybe it's better to post it here https://github.com/AviSynth/AviSynthPlus/issues

[Dogway]

@Boulder: Test with this. I removed all pixel addressing calls, try to use the same threads than in STGMC, I don't know what's the default there.

[Boulder]

Originally Posted by Dogway

@Boulder: Test with this. I removed all pixel addressing calls, try to use the same threads than in STGMC, I don't know what's the default there.

There's a function missing:
error: Script error: There is no function named 'QTGMC_deflate'.
(C:/Program Files/AviSynth+/plugins64/qtgmc.avsi, line 1064)
(C:/Program Files/AviSynth+/plugins64/qtgmc.avsi, line 907)

[Boulder]

The issue might be partly related to SRestore as well. Running it in HBD also causes the leak with STGMC. It would be interesting to see if the tuned version of QTGMC handles 8-bits though, since the original version didn't. I'll doublecheck that one more time though.

SetMemoryMax(8192)
DGSource("source.dgi") # 1080i, 29.97 fps
STGMC(preset="medium", noisepreset="fast", tr2=0, sigma=0.25)
ConvertBits(16)
Spline36Resize(1280,720)
SRestore()

[Dogway]

Yes sorry, here are both functions from old QTGMC:

# Functions (from original TGMC) used instead of mt_xxflate with similar operation but a somewhat stronger result. Originally added for speed, they are
# no longer faster due to improvements in masktools. Difference (visual and speed) is small so may be reverted in a later version.
function QTGMC_inflate( clip c, int "Y", int "U", int "V" )
{
	# >>>> YUY2 is planar here
	mtY =default( Y, 3 )
	mtU =default( U, 1 )
	mtV =default( V, 1 )
	rgY = (mtY == 3) ? 20 : -1
	rgU = (mtU == 3) ? 20 : -1
	rgV = (mtV == 3) ? 20 : -1
	mt_logic( c, c.RemoveGrain( rgY, rgU, rgV ), "max", Y=mtY,U=mtU,V=mtV )
}

function QTGMC_deflate( clip c, int "Y", int "U", int "V" )
{
	# >>>> YUY2 is planar here
	mtY =default( Y, 3 )
	mtU =default( U, 1 )
	mtV =default( V, 1 )
	rgY = (mtY == 3) ? 20 : -1
	rgU = (mtU == 3) ? 20 : -1
	rgV = (mtV == 3) ? 20 : -1
	mt_logic( c, c.RemoveGrain( rgY, rgU, rgV ), "min", Y=mtY,U=mtU,V=mtV )
}

If STGMC also shows a memory leak in HBD (16-bit I understand) it could be a LUT problem. LUT calculations, at least for 16-bit are enabled for ex_lut()/mt_lut() only.

One more thing, it has been a long time I used srestore() but as far as I recall it was very picky with frame access. I used to use it only with MP_Pipeline() for these reasons or at another pass. You can also try to put RequestLinear() before the call.

[Dogway]

On another note, I coded a few new filters for ExTools, usually this involves several iterations that don't make the cut to the final version but there still might be some value on them.
This time it happened with Harris corner detection, I created one version that differs from the canonical one from the papers but IMO performs better.
So here it is the "by the book" Harris corner detection with helper function Non-Maximum Suppression.

# Harris corner detection
#
# Alternative version which works well for small images and graphics
function ex_corner_canon(clip a, float "sigma", float "th", bool "show") {

    rgb = isRGB(a)
    isy = isy(a)
    bi  = BitsPerComponent(a)

    rd = Default(sigma,    1.4)
    th = Default(th,       0.7) # From 0.1 to 0.9
    sh = Default(show,  false)
    th = th/10000.

    a = ConvertBits(a,32)
    a

    x1 = ex_lut("x[0,-1] x[0,1] - dup *").ex_blur(rd,mode="gaussian")
    y1 = ex_lut("x[-1,0] x[1,0] - dup *").ex_blur(rd,mode="gaussian")
    xy = ex_lutxy(x1,y1,"x y * ").ex_blur(rd,mode="gaussian")
#    xy = ex_lut("x[0,-1] x[0,1] - abs x[-1,0] x[1,0] - abs max").ex_blur(rd,mode="gaussian")

    ex_lutxyz(x1,y1,xy,"x y * z - x y + dup * 0.05 * -",UV=2) # R = Det(M) - k*Trace(M)^2
    ex_NMS(thin=1.0) # In the paper this goes after thresholding, but this resulted better on my tests
    ex_lut(Format("x {th} > range_max 0 ?"),UV=2)
    # Alternative without suppression
#    ex_lutxyz(x1,y1,xy,Format("x y * z -   x y + dup * 0.04 * - {th} > range_max 0 ?"),UV=2)

    # Corner augment
    ex_expand(3,mode="disk").ex_deflate()

    # Show
    sh ? ex_blend(a,last, "linearadd", 0.4, 1.0) : last
    ConvertBits(bi) }


# Non-Maximum Suppression (NMS) - Edge Thinning Algorithm
#
# ex_NMS(thin=0.3) # 15.5 (164 without scaling)
# awarpsharp2(chroma=2, depth=12, blur=4, thresh=128) # 167
# aWarpSharp(depth=8,blurlevel=2,thresh=0.5) # 182
#
function ex_NMS(clip a, float "thin", bool "show") {

    ss = Default(thin,  0.5) # Super sampling factor
    sh = Default(show,  false)
    ss = min(4.0,1./ss)

    RatioResize(a,ss,"%",kernel="blackman",src_left=0.25,src_top=0)

    ex_lut("x[0,-1] T@ x[0,1] B@ -
            x[-1,0] L@ x[1,0] R@ -
            atan2 0.1591549431 * dup 0 > swap dup 1 + ? 180 * A@

              157.5 > x[0,0] X@ T > X B > & X L ?
            A 112.5 > X x[1,-1] UR@ > X x[-1,1] DL@ > & X  DL ?
            A  67.5 > X L > X R > & X  T  ?
                      X x[-1,-1] UL@ > X x[1,1] > & X UL ? ? ? ? ",UV=2)

    RatioResize(1./ss,"%",kernel="bicubic",b=0,c=1)

    ex_limitdif(a, last, thr=2, elast=5, UV=2) }

[Boulder]

I think I found the culprit after many rounds of debugging - it's propCopy and that's why your QTGMC leaks memory since STGMC doesn't use the props feature at all.

This won't stall:

DGSource("source.dgi")
STGMC(preset="medium", noisepreset="fast", tr2=0, sigma=0.25)
propSet("_FieldBased", 0)
Spline36Resize(1280,720)
SRestore()

While this will:

DGSource("source.dgi")
org=last
STGMC(preset="medium", noisepreset="fast", tr2=0, sigma=0.25)
propCopy(last, org)
propSet("_FieldBased", 0)
Spline36Resize(1280,720)
SRestore()

I think I need to report this in GitHub along with the other property related issue I found earlier.

[Dogway]

Oh, that one was also reported back in October, still no word from pinterf.

So you are saying that my QTGMC mod without propCopy is fine? Or do you want to keep debugging it?

Can someone explain why this is happening...

I have a PC with an old AMD R9 290 GPU (which DOES NOT support CUDA), and when I use BM3D CUDA, of course it fails, BUT, I have also installed a basic GTX 650 (which DOES support CUDA).

And now, if I use BM3DCUDA, both are encoding, using OpenCL.

The R9 290 is a LOT faster as it has 40 compute units, whereas the poor little GTX 650 only has 2.

[poisondeathray]

Originally Posted by TDS

Can someone explain why this is happening...

I have a PC with an old AMD R9 290 GPU (which DOES NOT support CUDA), and when I use BM3D CUDA, of course it fails, BUT, I have also installed a basic GTX 650 (which DOES support CUDA).

And now, if I use BM3DCUDA, both are encoding, using OpenCL.

The R9 290 is a LOT faster as it has 40 compute units, whereas the poor little GTX 650 only has 2.

GTX 650 is too old

https://github.com/WolframRhodium/VapourSynth-BM3DCUDA

Requirements

CUDA-enabled GPU(s) of compute capability 5.0 or higher (Maxwell+).

Originally Posted by poisondeathray

GTX 650 is too old

https://github.com/WolframRhodium/VapourSynth-BM3DCUDA

I don't care about requirements, it just WORKS !!!!!!

[poisondeathray]

Originally Posted by TDS

I don't care about requirements, it just WORKS !!!!!!

ahh I misunderstood;

Maybe CPU fallback ? What were the CPU's between the 2 systems?

How are you measuring OpenCL usage ? What encoder are you using ? (is it using OpenCL as well) ?