Log in

View Full Version : Mdegrain with SVPflow problem


Pages : [1] 2

anton_foy
3rd December 2021, 18:27
Hello I try to use Mdegrain (in HBD 16-bit) but with SVPflow's SVAnalyse and SVConvert feeding it with the clip converted to 8-bit (only for the analysing part) but I get an error Invalid arguments to 'Mdegrain3'
Here is the script:
function FastDegrain_svp( clip src, int "degrain", int "blksize", int "overlap", int "pel",
\ int "thSAD", int "limit")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, 2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)

# Create our Super clip for MV search. Then, search for motion vectors.
src8 = src.convertbits(8).converttoyv12()

super = SVSuper(src8,"{gpu:1}")
v3 = super.SVAnalyse("{block:{w:8,overlap:2},special:{delta:3}}")
v2 = super.SVAnalyse("{block:{w:8,overlap:2},special:{delta:2}}")
v1 = super.SVAnalyse("{block:{w:8,overlap:2}}")

bv3 = (degrain>=3) ? v3.SVConvert(isb=true) : NOP()
bv2 = (degrain>=2) ? v2.SVConvert(isb=true) : NOP()
bv1 = (degrain>=1) ? v1.SVConvert(isb=true) : NOP()
fv1 = (degrain>=1) ? v1.SVConvert(isb=false) : NOP()
fv2 = (degrain>=2) ? v2.SVConvert(isb=false) : NOP()
fv3 = (degrain>=3) ? v3.SVConvert(isb=false) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
sourceSuper = src.MSuper(pel=pel, hpad=0, vpad=0)
src = (degrain==1) ? src.MDegrain1(super, sourceSuper, bv1, fv1, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(super, sourceSuper, bv1, fv1, bv2, fv2, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(super, sourceSuper, bv1, fv1, bv2, fv2, bv3, fv3, limit=limit) : src

return src
}

fastdegrain_spv(degrain=3)


I cannot figure out what is wrong, any ideas? Thanks!

EDIT: I also do not know how to script the parameters for defaults etc. for blocksize and overlap but I leave this for later.

StainlessS
3rd December 2021, 21:14
How do you have SVP stuff set up ?

Current dll's ?, [EDIT: svpflow-4.3.0.168 :- http://avisynth.nl/index.php/SVPFlow ]
and somewhere it says it needs a manager, no idea what that means,
does it mean SubJunk script Interframe2.avsi ?

EDIT: That web address thingy SVP uses, always kinda freaked me out a bit.

EDIT: from Svpflow link
As an alternative you can you use version 4.2.0.142 which does not have the SVP Manager requirement.
If you use that one, can you upload this somewhere [eg SendSpace.com {No account needed} ::: My ISP dont let me go to Web.Archive.org]
https://web.archive.org/web/20190322064557/http://www.svp-team.com/files/gpl/svpflow-4.2.0.142.zip

anton_foy
4th December 2021, 12:15
How do you have SVP stuff set up ?

Current dll's ?, [EDIT: svpflow-4.3.0.168 :- http://avisynth.nl/index.php/SVPFlow ]
and somewhere it says it needs a manager, no idea what that means,
does it mean SubJunk script Interframe2.avsi ?

EDIT: That web address thingy SVP uses, always kinda freaked me out a bit.

EDIT: from Svpflow link

If you use that one, can you upload this somewhere [eg SendSpace.com {No account needed} ::: My ISP dont let me go to Web.Archive.org]
https://web.archive.org/web/20190322064557/http://www.svp-team.com/files/gpl/svpflow-4.2.0.142.zip

Thanks StainlessS! I will try this when Im back this evening and I will upload the file for you to download.

EDIT:
Here is the file https://www.sendspace.com/file/3wur36
It still doesn't work for me, same error (in full) Mp_pipeline: Unable to create slave process. Message: Script error:
Invalid arguments to function 'MDegrain3'.

Selur
4th December 2021, 13:37
Is current mvtools2 still backward compatible with mvtools 2.5?

StainlessS
4th December 2021, 15:05
Is current mvtools2 still backward compatible with mvtools 2.5?
So far as I know, yes - excepting that current mvtools has separate XP compatible version dll's .

EDIT: Oops, and thanks for posting the link anton_foy.

Selur
4th December 2021, 16:37
Looking at the documentation
MDeGrain1 (clip, clip super, clip mvbw, clip mvfw, int "thSAD", int "thSADC", int "plane", int "limit", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
and looking at:
src.MDegrain1(super, sourceSuper, bv1, fv1, limit=limit)
isn't it wrong? Looks to me like src&super both are assigned to the clip variable of MDeGrain1,....

Boulder
4th December 2021, 18:04
Yes, that's definitely a problem there. Just one super clip is required.

StainlessS
4th December 2021, 18:12
Yep, eagle eyed Selur seems to have spotted OP problem [I looked and yet did not see :( ].

EDIT: And plus points for boulder too. :)

Selur
4th December 2021, 18:31
Hmm,..
still overlooking something, error is still the same when using:

function FastDegrain_svp( clip src, int "degrain", int "blksize", int "overlap", int "pel", int "thSAD", int "limit")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, 2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)

# Create our Super clip for MV search. Then, search for motion vectors.
src8 = src.convertbits(8).converttoyv12()
super8 = SVSuper(src8, "{gpu:1}")

v1 = SVAnalyse(super8, "{block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:1}}}")
bv1 = SVConvert(v1, isb=true) # SVConvert: forward <> isb=false, backward isb=true
fv1 = SVConvert(v1, isb=false)

v2 = SVAnalyse(super8, "{block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:2}}")
bv2 = SVConvert(v2, isb=true)
fv2 = SVConvert(v2, isb=false)

v3 = SVAnalyse(super8, "{block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:3}}")
bv3 = SVConvert(v3, isb=true)
fv3 = SVConvert(v3, isb=false)
# Degraining the video using MVDegrain.

# MDeGrain1 (clip, clip super, clip mvbw, clip mvfw, int "thSAD", int "thSADC", int "plane", int "limit", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
src = (degrain > 0) ? MDegrain1(clip=src8, super=super8, mvbw=bv1, mvfw=fv3, thSAD=thSAD, limit=limit) : src8
# MDeGrain2 (clip, clip super, clip mvbw, clip mvfw, clip mvbw2, clip mvfw2, int "thSAD", int "thSADC", int "plane", int "limit", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
src = (degrain > 1) ? MDegrain2(clip=src8, super=super8, mvbw=bv1, mvfw=fv1, mvbw2=bv2, mvfw2=fv2, thSAD=thSAD, limit=limit) : src8
# MDeGrain3 (clip, clip super, clip mvbw, clip mvfw, clip mvbw2, clip mvfw2, clip mvbw3, clip mvfw3,int "thSAD", int "thSADC", int "plane", int "limit", int "limitC", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
src = (degrain > 2) ? MDegrain3(clip=src8, super=super8, mvbw=bv1, mvfw=fv1, mvbw2=bv2, mvfw2=fv2, mvbw3=bv3, mvfw=fv3, thSAD=thSAD, limit=limit) : src8

return src
}


Cu Selur

StainlessS
4th December 2021, 18:41
Posted a load of nonsense, and removed.
But, [from Selur post]
# MDeGrain1 (clip, clip super, clip mvbw, clip mvfw, int "thSAD", int "thSADC", int "plane", int "limit", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
src = (degrain > 0) ? MDegrain1(clip=src8, super=super8, mvbw=bv1, mvfw=fv3, thSAD=thSAD, limit=limit) : src8

Args clip, through to mvfw are NOT optional and cannot be called with named args. [Also check vector names eg fv3 for MDegrain1 (maybe should be fv1)].

Or like

src = (degrain > 0) ? MDegrain1(src8, super8, bv1,fv1, thSAD=thSAD, limit=limit) : src8


EDIT: I still dont have SVFlow stuff set up.

EDIT: So maybe something like this [untested]

function FastDegrain_svp( clip src, int "degrain", int "blksize", int "overlap", int "pel",
\ int "thSAD", int "limit")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, 2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)

# Create our Super clip for MV search. Then, search for motion vectors.
src8 = src.convertbits(8).converttoyv12()

superSV = SVSuper(src8,"{gpu:1}")
v3 = superSV.SVAnalyse("{block:{w:8,overlap:2},special:{delta:3}}")
v2 = superSV.SVAnalyse("{block:{w:8,overlap:2},special:{delta:2}}")
v1 = superSV.SVAnalyse("{block:{w:8,overlap:2}}")

bv3 = (degrain>=3) ? v3.SVConvert(isb=true) : NOP()
bv2 = (degrain>=2) ? v2.SVConvert(isb=true) : NOP()
bv1 = (degrain>=1) ? v1.SVConvert(isb=true) : NOP()
fv1 = (degrain>=1) ? v1.SVConvert(isb=false) : NOP()
fv2 = (degrain>=2) ? v2.SVConvert(isb=false) : NOP()
fv3 = (degrain>=3) ? v3.SVConvert(isb=false) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
sourceSuper = src.MSuper(pel=pel, hpad=0, vpad=0)
src = (degrain==1) ? src.MDegrain1(sourceSuper, bv1, fv1, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(sourceSuper, bv1, fv1, bv2, fv2, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(sourceSuper, bv1, fv1, bv2, fv2, bv3, fv3, limit=limit) : src

return src
}

fastdegrain_spv(degrain=3)


I renamed Super to superSV.

EDIT: Added arg GPU, where calls superSV = SVSuper(src8,"{gpu:0}") when arg gpu=false {I presume that is correct, test it}.


function FastDegrain_svp( clip src, int "degrain", int "blksize", int "overlap", int "pel",
\ int "thSAD", int "limit",Bool "gpu")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, 2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)
gpu = default( gpu, true ) # True, use GPU

# Create our Super clip for MV search. Then, search for motion vectors.
src8 = src.convertbits(8).converttoyv12()

superSV = SVSuper(src8,"{gpu:" + (gpu?"1":"0") + "}")
v3 = superSV.SVAnalyse("{block:{w:8,overlap:2},special:{delta:3}}")
v2 = superSV.SVAnalyse("{block:{w:8,overlap:2},special:{delta:2}}")
v1 = superSV.SVAnalyse("{block:{w:8,overlap:2}}")

bv3 = (degrain>=3) ? v3.SVConvert(isb=true) : NOP()
bv2 = (degrain>=2) ? v2.SVConvert(isb=true) : NOP()
bv1 = (degrain>=1) ? v1.SVConvert(isb=true) : NOP()
fv1 = (degrain>=1) ? v1.SVConvert(isb=false) : NOP()
fv2 = (degrain>=2) ? v2.SVConvert(isb=false) : NOP()
fv3 = (degrain>=3) ? v3.SVConvert(isb=false) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
sourceSuper = src.MSuper(pel=pel, hpad=0, vpad=0)
src = (degrain==1) ? src.MDegrain1(sourceSuper, bv1, fv1, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(sourceSuper, bv1, fv1, bv2, fv2, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(sourceSuper, bv1, fv1, bv2, fv2, bv3, fv3, limit=limit) : src

return src
}

fastdegrain_spv(degrain=3)

Selur
4th December 2021, 19:48
Args clip, through to mvfw are NOT optional and cannot be called with named args. [Also check vector names eg fv3 for MDegrain1 (maybe should be fv1)].
Argh,..

Okay:
function FastDegrain_svp( clip src, int "degrain", int "blksize", int "overlap", int "pel", int "thSAD", int "limit",Bool "gpu")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, 2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)
gpu = default( gpu, true ) # True, use GPU

# Create our Super clip for MV search. Then, search for motion vectors.
src8 = src.convertbits(8).converttoyv12()

superSV = SVSuper(src8,"{gpu:" + (gpu?"1":"0") + "}")
v1 = (degrain>=1) ? superSV.SVAnalyse("{block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:1}}"): NOP()
v2 = (degrain>=2) ? superSV.SVAnalyse("{block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:2}}") : NOP()
v3 = (degrain>=3) ? superSV.SVAnalyse("{block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:3}}") : NOP()

bv1 = (degrain>=1) ? v1.SVConvert(isb=true) : NOP()
bv2 = (degrain>=2) ? v2.SVConvert(isb=true) : NOP()
bv3 = (degrain>=3) ? v3.SVConvert(isb=true) : NOP()

fv1 = (degrain>=1) ? v1.SVConvert(isb=false) : NOP()
fv2 = (degrain>=2) ? v2.SVConvert(isb=false) : NOP()
fv3 = (degrain>=3) ? v3.SVConvert(isb=false) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
sourceSuper = src.MSuper(pel=pel, hpad=0, vpad=0)
src = (degrain==1) ? src.MDegrain1(sourceSuper, bv1, fv1, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(sourceSuper, bv1, fv1, bv2, fv2, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(sourceSuper, bv1, fv1, bv2, fv2, bv3, fv3, limit=limit) : src

return src
}
gives: 'MeDgainX: wrong source or super frame size', something is still amiss.

src is 640x352
sourceSuper is 640x696
not sure what's expected,...

Regarding gpu, SVAnalyse should also use the gpu, so:
v1 = (degrain>=1) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:1}}"): NOP()
v2 = (degrain>=2) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:2}}") : NOP()
v3 = (degrain>=3) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:3}}") : NOP() should probably be used.


Cu Selur

StainlessS
4th December 2021, 20:32
function FastDegrain_svp( clip src, int "degrain", int "blksize", int "overlap", int "pel", int "thSAD", int "limit",Bool "gpu") {
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, 2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)
gpu = default( gpu, true ) # True, use GPU

# Create our Super clip for MV search. Then, search for motion vectors.
src8 = src.convertbits(8).converttoyv12()

superSV = SVSuper(src8,"{gpu:" + (gpu?"1":"0") + "}")
v1 = (degrain>=1) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:1}}") : NOP()
v2 = (degrain>=2) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:2}}") : NOP()
v3 = (degrain>=3) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:3}}") : NOP()

bv1 = (degrain>=1) ? v1.SVConvert(isb=true) : NOP()
bv2 = (degrain>=2) ? v2.SVConvert(isb=true) : NOP()
bv3 = (degrain>=3) ? v3.SVConvert(isb=true) : NOP()

fv1 = (degrain>=1) ? v1.SVConvert(isb=false) : NOP()
fv2 = (degrain>=2) ? v2.SVConvert(isb=false) : NOP()
fv3 = (degrain>=3) ? v3.SVConvert(isb=false) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
sourceSuper = src.MSuper(pel=pel, hpad=0, vpad=0)

RT_DebugF("Src.w=%d Src.H=%d",Src.Width,Src.Height)
RT_DebugF("bv1.w=%d bv1.H=%d",bv1.Width,bv1.Height)
RT_DebugF("fv1.w=%d fv1.H=%d",fv1.Width,fv1.Height)
RT_DebugF("sourceSuper.w=%d sourceSuper.H=%d",sourceSuper.Width,sourceSuper.Height)

src = (degrain==1) ? src.MDegrain1(sourceSuper, bv1, fv1, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(sourceSuper, bv1, fv1, bv2, fv2, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(sourceSuper, bv1, fv1, bv2, fv2, bv3, fv3, limit=limit) : src

return src
}

LoadPlugin(".\svpflow1_64.dll")
LoadPlugin(".\svpflow2_64.dll")

BlankClip(Width=640,height=352,pixel_type="YV12")

FastDegrain_svp(degrain=1)



00000169 19:30:49 RT_DebugF: Src.w=640 Src.H=352
00000170 19:30:49 RT_DebugF: bv1.w=18511 bv1.H=1
00000171 19:30:49 RT_DebugF: fv1.w=18511 fv1.H=1
00000172 19:30:49 RT_DebugF: sourceSuper.w=640 sourceSuper.H=696

Not sure, I think the Super clip consists of stacked full size + half size + quarter size + ... {about 5 sizes with default levels=0 meaning all levels}
Weird in RED [not sure what should come out].

EDIT: OK, Added your gpu edit stuff, same result to debugview.

Boulder
4th December 2021, 20:40
What's pel in the first super clip (for analysis)? Is the default 2?

Selur
4th December 2021, 21:13
according to https://www.svp-team.com/wiki/Manual:SVPflow#Basic_Vapoursynth_script default is 2
-> works with:
function FastDegrain_svp( clip src, int "degrain", int "blksize", int "overlap", int "pel", int "thSAD", int "limit",Bool "gpu")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, 2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)
gpu = default( gpu, true ) # True, use GPU

# Create our Super clip for SVP MV search. Then, search for motion vectors.
src8 = src.convertbits(8).converttoyv12()

superSV = SVSuper(src8,"{gpu:" + (gpu?"1":"0") + ",pel:"+string(pel)+"}")
v1 = (degrain>=1) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:1}}"): NOP()
v2 = (degrain>=2) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:2}}") : NOP()
v3 = (degrain>=3) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:3}}") : NOP()

bv1 = (degrain>=1) ? v1.SVConvert(isb=true) : NOP()
bv2 = (degrain>=2) ? v2.SVConvert(isb=true) : NOP()
bv3 = (degrain>=3) ? v3.SVConvert(isb=true) : NOP()

fv1 = (degrain>=1) ? v1.SVConvert(isb=false) : NOP()
fv2 = (degrain>=2) ? v2.SVConvert(isb=false) : NOP()
fv3 = (degrain>=3) ? v3.SVConvert(isb=false) : NOP()

# Degraining the video using MVDegrain using vectors calculated through SVP
#sourceSuper = src.MSuper(pel=pel, hpad=0, vpad=0)
sourceSuper = src.MSuper(pel=pel, hpad=0, vpad=0)
src = (degrain==1) ? src.MDegrain1(sourceSuper, bv1, fv1, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(sourceSuper, bv1, fv1, bv2, fv2, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(sourceSuper, bv1, fv1, bv2, fv2, bv3, fv3, limit=limit) : src

return src
}
Nice :)

StainlessS
4th December 2021, 21:15
Pel default 2 in both MSuper and SvSuper.

MvTools:- http://avisynth.nl/index.php/MVTools
SvFlow:- http://avisynth.nl/index.php/SVPFlow

StainlessS
4th December 2021, 21:26
So problem was caused because of mismatch between Svp pel (default 2, was not set in call) and MSuper pel (default to the script function of 1).
Yes ?

Selur
4th December 2021, 21:44
So problem was caused because of mismatch between Svp pel (default 2, was not set in call) and MSuper pel (default to the script function of 1).
Yes. + In difference to the start, gpu, blksize and overlay are properly used. :)

Cu Selur

anton_foy
5th December 2021, 00:05
Amazing work guys! I really need to try it out when I come home. Thanks again!

Edit: btw. did you use the 4.2.0.142-version?

StainlessS
5th December 2021, 01:26
Edit: btw. did you use the 4.2.0.142-version?
Well when I did eventually get it setup, yes, thats what I used.

Seems that the other [latest] one requires SVP Manager, which is a pay for thing [$19.00 per machine, lifetime, I think - or 30 day trial - Linux ver$ is Free].
EDIT: As Here:- https://www.svp-team.com/get/

EDIT: $19.00 1st machine, thereafter $10.00 per machine.

Selur
5th December 2021, 07:04
did you use the 4.2.0.142-version?
nope I used:
svpflow1_64 version 4.30.156
and
svpflow2_64 version 4.3.0.161
both work without the SVP Manager. (see: https://forum.doom9.org/showthread.php?t=181379)

Cu Selur

Boulder
5th December 2021, 10:06
I actually don't understand why pel needs to be the same for both super clips. Aren't the vectors in MDegrain used only on one level that is actually the source without any upsizing?

anton_foy
5th December 2021, 14:40
Fastdegrain(degrain=3) 0.0222x speed and gives good denoise and most noise is removed.
Fastdegrain_svg(degrain=3) 0.0245x speed but too little noise removed in comparison to the mvtools only fastdegrain.

Also fastdegrain_svp(overlap=1) gives me a pink and green image.

StainlessS
5th December 2021, 14:42
I actually don't understand why pel needs to be the same for both super clips. Aren't the vectors in MDegrain used only on one level that is actually the source without any upsizing?

I tried this

Src=BlankClip(Width=640,height=352,pixel_type="YV12")

super1 = src.MSuper(pel=1)
super2 = src.MSuper(pel=2)
bv = super1.MAnalyse(isb=true )
fv = super1.MAnalyse(isb=false)

src.MDegrain1(super2, bv, fv)

MDegrainX: wrong source or super frame size.

EDIT: Swapping super1/super2 arg to pel gives same result.

EDIT:
Fastdegrain(degrain=3) 0.0222x speed and gives good denoise and most noise is removed.
where is FastDegrain() ?

EDIT:
Would a script function [using RT_Stats RT_String()] be of use, that creates the JSON parameter string things for
the SVP functions, would need separate script func for each SVP func.
[would take same args as SVP stuff, & use SVP defaults where not supplied]

Selur
5th December 2021, 14:57
Also fastdegrain_svp(overlap=1) gives me a pink and green image.
Not really surprising, since there's a difference between what overlap does in SVP and mvtools.

SVAnalyse, overlap:
Block overlap value. 0 - none, 1 - 1/8 of block size in each direction, 2 - 1/4 of block size, 3 - 1/2 of block size. The greater overlap, the more blocks number, and the lesser the processing speed.

Resulting overlap value in pixels should be even with CPU rendering. source: https://www.svp-team.com/wiki/Manual:SVPflow#SVAnalyse.28super.2C_params_string.2C_.5Bsrc.5D:_clip.29
while
MAnalyse, overlap:
overlap: block overlap value (horizontal). Must be even and less than block size (up to blksize/2 for MCompensate). The step between blocks for motion estimation is equal to (blksize-overlap). N blocks cover the size ((blksize-overlap)*N + overlap) on frame. Try use overlap value from blksize/4 to blksize/2. The greater overlap, the more blocks number, and the lesser the processing speed. Default value is 0.
Functions with overlap support are: MFlow, MFlowInter, MFlowFps, MShow, MMask, MCompensate, MDeGrain1, MDeGrain2, MDeGrain3. source: http://avisynth.org.ru/mvtools/mvtools2.html

-> you will probably have to adjust the overlap value in the mvtools calls accordingly.

Cu Selur

StainlessS
5th December 2021, 15:20
I actually don't understand why pel needs to be the same for both super clips. Aren't the vectors in MDegrain used only on one level that is actually the source without any upsizing?
EDIT:
Levels in MSuper,control the number of downsized images [halved at each level] in super clip.
MAnalyse uses these multiple levels to aid in faster searching [smallest first and bigger for more precise - and pel comes into it somewhere for more precision], results in vector clip.
MDegrain, creates candidate frames by moving current frame [+/- delta] blocks along vectors with perhaps sub pixel accuracy [super clip pel], and uses various blending stuff
based on SAD and other [eg exclusion] masks to temporally soften the grain of the original frame.
Where MDegrain creates candidate frame and masks result in a bad block, then block from original frame used [perhaps with blur].
MDegrain good candidate frame blocks will be created using the vectors clips AND MSuper PEL to subpixel accuracy.
Only MAnalyse requires MSuper mutliple levels clip, I think [just for the search - dont think levels has any other use].
Dont think there is any upsizing [unless its used in the subpixel candidate frame creation and averaging]. [if you know different, then do tell - above ALL guesswork and a bit muddled].
END_EDIT:

The answer is hidden amongst this lot I think.


Src=BlankClip(Width=640,height=352,pixel_type="YV12")

super1 = src.MSuper(pel=1)
bv1 = super1.MAnalyse(isb=true )
fv1 = super1.MAnalyse(isb=false)

super2 = src.MSuper(pel=1)
bv2 = super2.MAnalyse(isb=true )
fv2 = super2.MAnalyse(isb=false)

RT_DebugF("Super1.w=%d Super1.H=%d",Super1.Width,Super1.Height)
RT_DebugF("bv1.w=%d bv1.H=%d",bv1.Width,bv1.Height)

RT_DebugF("Super2.w=%d Super2.H=%d",Super2.Width,Super2.Height)
RT_DebugF("bv2.w=%d bv2.H=%d",bv2.Width,bv2.Height)
Src



00000169 14:04:26 RT_DebugF: Super1.w=656 Super1.H=832
00000170 14:04:26 RT_DebugF: bv1.w=14118 bv1.H=1
00000171 14:04:26 RT_DebugF: Super2.w=656 Super2.H=832
00000172 14:04:26 RT_DebugF: bv2.w=14118 bv2.H=1

Everything same.

Pel is hidden as value in super somewhere.

and maybe extract some hint from [mainly the edit at the end]

From here:- https://forum.doom9.org/showthread.php?p=1840183#post1840183
Post #16 script updated.

Shows colors of the vector greys, and little white (single pixel) dot overlaid where taken from (Align MUST be true otherwise not shown).

eg

CS="Y8"
#CS="YV24"
VectorTest(Delta=2,time=50.0,HFlip=False,CS=CS)


https://s20.postimg.cc/vnubrzazx/Vector_Test.jpg (https://postimages.org/)
Fwd/Bak, Is mostly opposite (relative mid grey, 128), but sometimes goes awry (usually at edge of grey, we try take mid pixel).

EDIT: With CS="YV24"
https://s20.postimg.cc/6vupkxh0d/Vector_Test_Col.jpg (https://postimages.org/)

EDIT: The movement in above graphics should be DELTA(2) * XSTEP(8)=16, but in MSuper, Pel=2, ie 2*8*2=32,
so for the vector Greys = +/- 32 , seems about right.
[XSTEP is the horizontal pixel movement per frame, Delta is the number of frames apart, and Pel=2 is at 1/2 pixel resolution].

So is something like Pel * vector length [H:V vector length & direction stored as color in the vector clip].
Below WRONG
[For 8 bit clip, vector length stored as signed 8 bit value - so very limited in range, sign = vector direction, and 7 bits only for length]
[of course the vector clips store 8 bit values for both horizontal and vertical direction of the vector - above graphic only for horizontal part of vector]
EDIT: NOTE ALSO, with 8 bit src clip, only 7 bits for vector length, so any movement greater than 127 pixels cannot be stored in 8 bit vector clip, so that vector and resulting block bad (I think).

anton_foy
5th December 2021, 16:13
where is FastDegrain() ?


Sorry, this one:
function FastDegrain( clip src, int "degrain", int "blksize", int "overlap", int "pel",
\ int "thSAD", int "limit")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, blksize/2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)

# Create our Super clip for MV search. Then, search for motion vectors.
super = src.MSuper(pel=pel)
bvec3 = (degrain>=3) ? super.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap) : NOP()
bvec2 = (degrain>=2) ? super.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap) : NOP()
bvec1 = (degrain>=1) ? super.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap) : NOP()
fvec1 = (degrain>=1) ? super.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap) : NOP()
fvec2 = (degrain>=2) ? super.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap) : NOP()
fvec3 = (degrain>=3) ? super.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, limit=limit) : src

return src
}

Thanks Selur, I should have looked up Overlap more closely between them.

StainlessS
5th December 2021, 16:41
Thanks.

EDIT: Time to feed the ducks.

Selur
5th December 2021, 17:02
Thanks Selur, I should have looked up Overlap more closely between them.
you are welcome, btw. you don't need to adjust the mvtools calls, you need to make sure that blksize is at least 16 when using (svp) overlap 1. ;)

hello_hello
5th December 2021, 17:06
Shouldn't FastDegrain be this? And the same for FastDegrain_svp ? (additions in blue)

function FastDegrain( clip src, int "degrain", int "blksize", int "overlap", int "pel",
\ int "thSAD", int "limit")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, blksize/2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)

# Create our Super clip for MV search. Then, search for motion vectors.
super = src.MSuper(pel=pel)
bvec3 = (degrain>=3) ? super.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap) : NOP()
bvec2 = (degrain>=2) ? super.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap) : NOP()
bvec1 = (degrain>=1) ? super.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap) : NOP()
fvec1 = (degrain>=1) ? super.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap) : NOP()
fvec2 = (degrain>=2) ? super.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap) : NOP()
fvec3 = (degrain>=3) ? super.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, thSAD=thSAD, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=limit) : src

return src
}

I appear to have modified my copy at some stage as I'm not a fan of truemotion (it tends to increase blurring) so I disabled it by default. There's also an argument for enabling/disabling internal multithreading. Additions in green.

function FastDegrain( clip src, int "degrain", int "blksize", int "overlap", int "pel",
\ int "thSAD", int "limit", bool "TrueM", bool "GlobalM", bool "MT")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, blksize/2 ) # MVAnalyse block overlap (even, ov<=blksize/2)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)
TrueM = default( TrueM, false ) # Enable/disable truemotion
GlobalM = default( GlobalM, true ) # Enable/disable global motion
MT = default( MT, true ) # Internal multithreading (through avstp.dll)

# Create our Super clip for MV search. Then, search for motion vectors.
super = src.MSuper(pel=pel)
bvec3 = (degrain>=3) ? \
super.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap, truemotion=TrueM, global=GlobalM, mt=MT) : NOP()
bvec2 = (degrain>=2) ? \
super.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap, truemotion=TrueM, global=GlobalM, mt=MT) : NOP()
bvec1 = (degrain>=1) ? \
super.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap, truemotion=TrueM, global=GlobalM, mt=MT) : NOP()
fvec1 = (degrain>=1) ? \
super.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap, truemotion=TrueM, global=GlobalM, mt=MT) : NOP()
fvec2 = (degrain>=2) ? \
super.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap, truemotion=TrueM, global=GlobalM, mt=MT) : NOP()
fvec3 = (degrain>=3) ? \
super.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap, truemotion=TrueM, global=GlobalM, mt=MT) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, thSAD=thSAD, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=limit) : src

return src
}

Selur
5th December 2021, 18:05
Yeah, thSAD got lost during a copy and paste.
So FastDegrain_svp should be:
function FastDegrain_svp( clip src, int "degrain", int "blksize", int "overlap", int "pel", int "thSAD", int "limit",Bool "gpu")
{
degrain = default( degrain, 2 ) # Degraining method (1 - 3)
blksize = default( blksize, 8 ) # MAnalyse block size (4, 8, 16)
overlap = default( overlap, blksize/2 ) # MVAnalyse block overlap (0 - none, 1 - 1/8 of block size in each direction, 2 - 1/4 of block size, 3 - 1/2 of block size.)
pel = default( pel, 1 ) # MVAnalyse pel (1, 2, 4)
thSAD = default( thSAD, 400 ) # MVDegrain thSAD
limit = default( limit, 255 ) # MVDegrain limit (0 - 255)

# Create our Super clip for SVP MV search. Then, search for motion vectors.
src8 = src.convertbits(8).converttoyv12()

superSV = SVSuper(src8,"{gpu:" + (gpu?"1":"0") + ",pel:"+string(pel)+"}")
v1 = (degrain>=1) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:1}}"): NOP()
v2 = (degrain>=2) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:2}}") : NOP()
v3 = (degrain>=3) ? superSV.SVAnalyse("{gpu:" + (gpu?"1":"0") + ",block:{w:"+string(blksize)+",overlap:"+string(overlap)+"},special:{delta:3}}") : NOP()

bv1 = (degrain>=1) ? v1.SVConvert(isb=true) : NOP()
bv2 = (degrain>=2) ? v2.SVConvert(isb=true) : NOP()
bv3 = (degrain>=3) ? v3.SVConvert(isb=true) : NOP()

fv1 = (degrain>=1) ? v1.SVConvert(isb=false) : NOP()
fv2 = (degrain>=2) ? v2.SVConvert(isb=false) : NOP()
fv3 = (degrain>=3) ? v3.SVConvert(isb=false) : NOP()

# Degraining the video using MVDegrain using vectors calculated through SVP
sourceSuper = src.MSuper(pel=pel, hpad=0, vpad=0)
src = (degrain==1) ? src.MDegrain1(sourceSuper, bv1, fv1, thSAD=thSAD, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(sourceSuper, bv1, fv1, bv2, fv2, thSAD=thSAD, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(sourceSuper, bv1, fv1, bv2, fv2, bv3, fv3, thSAD=thSAD, limit=limit) : src

return src
}

StainlessS
5th December 2021, 18:13
HH,
ThSAD looks good.
TrueMotion, even if false then should maybe set global=true, almost always better. [Internally Global=truemotion also switches it off when false{global = setting for motion, track whole frame panning}].
mt=MT, think Pinterf might have commented on using MT setting, saying just makes things slower doing Prefetch in AVS+, and again internally in mvtools [default should maybe not by true, anyways].

anton_foy
5th December 2021, 18:22
HH,
ThSAD looks good.
TrueMotion, even if false then should maybe set global=true, almost always better. [Internally Global=truemotion also switches it off when false{global = setting for motion, track whole frame panning}].
mt=MT, think Pinterf might have commented on using MT setting, saying just makes things slower doing Prefetch in AVS+, and again internally in mvtools [default should maybe not by true, anyways].

Yes for me that is the case, when MT=True it is slower.

hello_hello
5th December 2021, 18:57
I recall QTGMC doing some fiddling when truemotion=false.

Now I remember. I added this to MCDegrainSharp way back then.
https://forum.doom9.org/showthread.php?p=1855907#post1855907

# Uncommenting the two lines below results in the same truemotion settings as QTGMC when truemotion=false
# lambda = default(lambda, (truemotion ? 1000 : 100 ) * BSize * BSize / 64)
# pnew = default(pnew, truemotion ? 50 : 25)

I think for MCDegrainSharp I tested the above settings and preferred the MVTools defaults, so they stayed commented out.

QTGMC has GlobalMotion=true as the default and it's not adjusted with Truemotion. I'm not sure how I missed that, but I probably should play around with GlobalMotion at some stage.

hello_hello
5th December 2021, 19:09
FastDegrain and FastDegrainSharp seem to have merged on the wiki, but it's still missing the thSAD argument for the MDegrains. Anyone with wiki access care to fix it?
http://avisynth.nl/images/FastDegrain.avs

Edit: I just remembered, there's also a mistake in line 44. This

src = (sharp==true) ? src.LimitedSharpenFaster(strength=str,ss_x=ss,ss_y=ss,soft=sft,overshoot=oshot) : src

should be

src = (sharp==true) ? src.LimitedSharpenFaster(strength=str,ss_x=ss,ss_y=ss,soft=sft,overshoot=oshoot) : src

StainlessS
5th December 2021, 19:21
So is something like Pel * vector length [H:V vector length & direction stored as color in the vector clip].
Below WRONG
[For 8 bit clip, vector length stored as signed 8 bit value - so very limited in range, sign = vector direction, and 7 bits only for length]
[of course the vector clips store 8 bit values for both horizontal and vertical direction of the vector - above graphic only for horizontal part of vector]
EDIT: NOTE ALSO, with 8 bit src clip, only 7 bits for vector length, so any movement greater than 127 pixels cannot be stored in 8 bit vector clip, so that vector and resulting block bad (I think).

Seems that vector clips do not store only byte sized values, seems that for 8 bit src vector coords [X and Y] are 2 bytes each.

(Also, when using mvtools to create visible vector clip [MShow I think], it can only be valid for 8 bit vectors [EDIT +ve/-ve values 0->127], dont know what happens if bigger than that)
EDIT: Above sorta wrong, visible vector clips created by MShow will be ok, vectors drawn over clip can be any lenth.
What I meant was for visible H and V vectors realized with MMask() where vector length can only be 8 bit same as result 8 bit clip.


W=640 # clip width
H=352 # clip height
BlkSz=8 # default

XBlks = (W+BlkSz-1)/BlkSz # Round UP number of whole blocks horizotnal
YBlks = (H+BlkSz-1)/BlkSz # Round UP number of whole blocks vertical
nBlks = XBlks*YBlks # total number of blocks [without overlapping blocks]
TotCoords = nBlks * 2 # Total number of vector cordinates [X, and Y, for each block]

Src=BlankClip(Width=W,height=H,pixel_type="YV12")

super1 = src.MSuper(pel=1,hpad=0,vpad=0)
bv1 = super1.MAnalyse(isb=true, BlkSize=BlkSZ,BlkSizeV=BlkSZ,Overlap=0,OverlapV=0)
fv1 = super1.MAnalyse(isb=false,BlkSize=BlkSZ,BlkSizeV=BlkSZ,Overlap=0,OverlapV=0)

super2 = src.MSuper(pel=1,hpad=0,vpad=0)
bv2 = super2.MAnalyse(isb=true, BlkSize=BlkSZ,BlkSizeV=BlkSZ,Overlap=0,OverlapV=0)
fv2 = super2.MAnalyse(isb=false,BlkSize=BlkSZ,BlkSizeV=BlkSZ,Overlap=0,OverlapV=0)

RT_DebugF("Super1.w=%d Super1.H=%d Super1.FrameCount=%d",Super1.Width,Super1.Height,Super1.FrameCount)
RT_DebugF("bv1.w=%d bv1.H=%d bv1.FrameCount=%d",bv1.Width,bv1.Height,bv1.FrameCount)

RT_DebugF("Super2.w=%d Super2.H=%d Super2.FrameCount=%d",Super2.Width,Super2.Height,Super2.FrameCount)
RT_DebugF("bv2.w=%d bv2.H=%d bv2.FrameCount=%d",bv2.Width,bv2.Height,bv2.FrameCount)

RT_DebugF("XBlks=%d YBlks=%d, nBlks=%d",XBlks,YBlks,nBlks) # blocks width, blocks tall, total number of blocks

RT_DebugF("TotCoords=%d",TotCoords)

bv1Bytes = bv1.width * bv1.height
RT_DebugF("bv1Bytes=%d",bv1Bytes)

BytesPerCoord = bv1bytes / TotCoords # Whole number of bytes per X or Y Coordinate
RT_DebugF("BytesPerCoord=%d",BytesPerCoord)

UnAccbytes = bv1Bytes - (BytesPerCoord * TotCoords) # Unaccounted for bytes in bv1
RT_DebugF("UnAccbytes=%d",UnAccbytes)

Src

result

00001099 18:14:13 RT_DebugF: Super1.w=640 Super1.H=696 Super1.FrameCount=240
00001100 18:14:13 RT_DebugF: bv1.w=14118 bv1.H=1 bv1.FrameCount=240
00001101 18:14:13 RT_DebugF: Super2.w=640 Super2.H=696 Super2.FrameCount=240
00001102 18:14:13 RT_DebugF: bv2.w=14118 bv2.H=1 bv2.FrameCount=240
00001103 18:14:13 RT_DebugF: XBlks=80 YBlks=44, nBlks=3520
00001104 18:14:13 RT_DebugF: TotCoords=7040
00001105 18:14:13 RT_DebugF: bv1Bytes=14118
00001106 18:14:13 RT_DebugF: BytesPerCoord=2
00001107 18:14:13 RT_DebugF: UnAccbytes=38

EDIT: Amongst the unaccounted for bytes in the vector clips, is maybe Pel [gotten from MSuper] and XBlks, and YBlks and overlaps, and H/V pad.

hello_hello
6th December 2021, 02:04
I haven't played around with it yet, but I've edited post #29 to add a global motion argument.

I did the same for my copy. I also added a FastDegrainSharp wrapper function so it can sharpen by default without FastDegrain having to. For anyone who might want it....

Edit: Whoops, I put the arguments for the FastDegrainSharp wrapper function in the wrong order. That's fixed.
Edit: I added some more options. See the next post.

# ===============================================================================
# ===============================================================================
# FastDegrain 2021-12-07
# ===============================================================================
# ===============================================================================
#
# FastDegrain by Sagekilla
# Easy to use alias for MDegrain 1, 2, or 3
# Features optional sharpening if you should desire
# Required plugins: LSFMod (to enable sharpening) & MVTools2.dll
#
# Changes by Hello_Hello
# Updated the required plugins list
# Added missing "thSAD" argument for MDegrain1, MDegrain2 & MDegrain3
# Added "TrueM" argument and made truemotion=false the default
# Added "GlobalM" argument and made global=true the default
# Added "MT" argument
# Added "Precise" argument
# LimitedSharpenFaster variable "oshot" corrected to "OShoot"
# Replaced LimitedSharpenFaster with LSFMod
# Added "SFirst" argument and moved the sharpening to after the degraining by default
# Changed the default for "Sharp" from 160 to 75
#
# -------------------------------------------------------------------------------
#
# Degrain: MVDegrain 1, 2, or 3. Each is increasingly stronger and slower.
# thSAD: Affects degraining: Low values reduce degraining, high values raise it.
# BLKSize: MVAnalsye blksize. See MVTools documentation for details.
# Overlap: MVAnalyse overlap. Must be even & <= BLKSize/2. Higher values are better & slower.
# Pel: MVAnalyse subpixel accuracy. Higher values give better MVs and lowers speed.
# Precise: Recalculate. It's quite a bit slower.
# Limit: Limits max MVDegrain change. Decrease if you experience artifacts.
# TrueM: Enables truemotion.
# GlobalM: Disables global motion.
# MT: Enables internal multithreading.
# Sharp: Enables LSFMod sharpening.
# Str: LSFMod's sharpening strength. Increase for more sharpening.
# SS: Supersampling for LSFMod, values over 1 decrease edge aliasing.
# Sft: Dampens LSFMod sharpening (it weakens the sharpening)
# OShoot: Sharpening limit, increase it too much and it causes haloing.
# SFirst: SFirst=true sharpens before denoising. The default is false.
#
# ===============================================================================
# ===============================================================================

function FastDegrain(clip Source, \
int "Degrain", int "BLKSize", int "thSAD", int "Overlap", int "Pel", bool "Precise", \
int "Limit", bool "TrueM", bool "GlobalM", int "lambda", int "pnew", bool "MT", \
bool "Sharp", int "Str", int "SS", int "Sft", int "OShoot", bool "SFirst") {

# -------------------------------------------------------------------------------

Degrain = default(Degrain, 2) # Degraining method (1 - 3)
thSAD = default(thSAD, 400) # MVDegrain thSAD
BLKSize = default(BLKSize, 8) # MAnalyse block size (4, 8, 16)
Overlap = default(Overlap, BLKSize/2) # MVAnalyse block overlap
Pel = default(Pel, 1) # MVAnalyse pel (1, 2, 4)
Precise = default(Precise, false) # Recalculate
Limit = default(Limit, 255) # MVDegrain limit (0 - 255)
TrueM = default(TrueM, false) # Enable/disable truemotion
GlobalM = default(GlobalM, true) # Enable/disable global motion
MT = default(MT, false) # Internal multithreading (through avstp.dll)
Sharp = default(Sharp, false) # Toggles LSFMod
Str = default(Str, 75) # LSFMod strength
SS = default(SS, 1.5) # LSFMod supersampling
Sft = default(Sft, 30) # LSFMod soft
OShoot = default(OShoot, 1) # LSFMod overshoot
SFirst = default(SFirst, false) # Enable sharpening before denoising

# Uncommenting the two lines below results in the same MVTools2 settings as QTGMC when truemotion=false
# lambda = default(lambda, (TrueM ? 1000 : 100 ) * BLKSize * BLKSize / 64)
# pnew = default(pnew, TrueM ? 50 : 25)

# -------------------------------------------------------------------------------

Input = !Sharp || !SFirst ? Source : \
Source.LSFMod(strength=Str, ss_x=SS, ss_y=SS, soft=Sft, overshoot=OShoot)

SuperClip = Input.MSuper(pel=Pel, mt=MT)

bvec3 = (Degrain >= 3) ? MAnalyse(SuperClip, isb=true, delta=3, blksize=BLKSize, overlap=Overlap, \
truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop()
bvec2 = (Degrain >= 2) ? MAnalyse(SuperClip, isb=true, delta=2, blksize=BLKSize, overlap=Overlap, \
truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop()
bvec1 = (Degrain >= 1) ? MAnalyse(SuperClip, isb=true, delta=1, blksize=BLKSize, overlap=Overlap, \
truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop()

fvec1 = (Degrain >= 1) ? MAnalyse(SuperClip, isb=false, delta=1, blksize=BLKSize, overlap=Overlap, \
truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop()
fvec2 = (Degrain >= 2) ? MAnalyse(SuperClip, isb=false, delta=2, blksize=BLKSize, overlap=Overlap, \
truemotion=TrueM, lambda=lambda, pnew=pnew, global=GlobalM, mt=MT) : nop()
fvec3 = (Degrain >= 3) ? MAnalyse(SuperClip, isb=false, delta=3, blksize=BLKSize, overlap=Overlap, \
truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT) : nop()

# -------------------------------------------------------------------------------

Precise ? Eval("""

bvec3 = (Degrain >= 3) ? MRecalculate(SuperClip, bvec3, blksize=BLKSize/2, overlap=Overlap/2, \
thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop()
bvec2 = (Degrain >= 2) ? MRecalculate(SuperClip, bvec2, blksize=BLKSize/2, overlap=Overlap/2, \
thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop()
bvec1 = (Degrain >= 1) ? MRecalculate(SuperClip, bvec1, blksize=BLKSize/2, overlap=Overlap/2, \
thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop()

fvec1 = (Degrain >= 1) ? MRecalculate(SuperClip, fvec1, blksize=BLKSize/2, overlap=Overlap/2, \
thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop()
fvec2 = (Degrain >= 2) ? MRecalculate(SuperClip, fvec2, blksize=BLKSize/2, overlap=Overlap/2, \
thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop()
fvec3 = (Degrain >= 3) ? MRecalculate(SuperClip, fvec3, blksize=BLKSize/2, overlap=Overlap/2, \
thSAD=thSAD/2, truemotion=TrueM, lambda=lambda, pnew=pnew) : nop()

""") : nop()

# -------------------------------------------------------------------------------

Output = \
(Degrain < 1) ? Input : \
(Degrain == 1) ? MDegrain1(Input, SuperClip, bvec1, fvec1, thSAD=thSAD, limit=Limit) : \
(Degrain == 2) ? MDegrain2(Input, SuperClip, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=Limit) : \
MDegrain3(Input, SuperClip, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=Limit)
Output = \
!Sharp || SFirst ? Output : Output.LSFMod(strength=Str, ss_x=SS, ss_y=SS, soft=Sft, overshoot=OShoot)

return Output }

# ===============================================================================
# ===============================================================================
# FastDegrainSharp
# ===============================================================================
# ===============================================================================

function FastDegrainSharp(clip Source, \
int "Degrain", int "BLKSize", int "thSAD", int "Overlap", int "Pel", bool "Precise", \
int "Limit", bool "TrueM", bool "GlobalM", int "lambda", int "pnew", bool "MT", \
bool "Sharp", int "Str", int "SS", int "Sft", int "OShoot", bool "SFirst") {

Sharp = default(Sharp, true)

FastDegrain(Source, Degrain, BLKSize, thSAD, Overlap, Pel, Precise, \
Limit, TrueM, GlobalM, lambda, pnew, MT, Sharp, Str, SS, Sft, OShoot, SFirst) }

# ===============================================================================
# ===============================================================================

hello_hello
6th December 2021, 21:23
I was experimenting with denoising a Looney Tunes Bluray today so I thought I'd try global motion. The source is so noisey I find it distracting so I'm trying to tone it down a bit, rather than get rid of it completely.

global=true definitely seems to be better than global=false. It's not a huge improvement but I think it's a little better.

I added lambda and pnew arguments to FastDegrain to see if the settings QTGMC uses when truemotion is disabled improves the quality when global=true. I still think the MVTools defaults for those two arguments are marginally better when truemotion=false.

I tried disabling MVTools' internal multithreading. I hadn't added prefetch to the script. MT=false was definitely slower than MT=true by a few fps when encoding the complete script I was using (including some other filters) but the output isn't identical and I think the quality takes a very slight hit with MT enabled.

While I was playing around I added a Precise argument to FastDegrain, although with Precise=true it's a NotSoFastDegrain. I'll edit my post above to replace the version of FastDegrain with the one I was playing with today, in case anyone wants it. The defaults are the same except for MT, which is disabled by default now.

anton_foy
7th December 2021, 01:46
I was experimenting with denoising a Looney Tunes Bluray today so I thought I'd try global motion. The source is so noisey I find it distracting so I'm trying to tone it down a bit, rather than get rid of it completely.

global=true definitely seems to be better than global=false. It's not a huge improvement but I think it's a little better.

I added lambda and pnew arguments to FastDegrain to see if the settings QTGMC uses when truemotion is disabled improves the quality when global=true. I still think the MVTools defaults for those two arguments are marginally better when truemotion=false.

I tried disabling MVTools' internal multithreading. I hadn't added prefetch to the script. MT=false was definitely slower than MT=true by a few fps when encoding the complete script I was using (including some other filters) but the output isn't identical and I think the quality takes a very slight hit with MT enabled.

While I was playing around I added a Precise argument to FastDegrain, although with Precise=true it's a NotSoFastDegrain. I'll edit my post above to replace the version of FastDegrain with the one I was playing with today, in case anyone wants it. The defaults are the same except for MT, which is disabled by default now.

Great work! In my script I use mp_pipeline and your previous version of fastdegrain with mt=true it is definitely slower on my computer than mt=false. Truemotion=true on my UHD-footage is looking horrible and unusable, no detail left. I will try global motion though. I was hoping the fastdegrain_SVP with gpu setting turned on would turn out faster (and it did a little bit) than fastdegrain mvtools version but the SVP-version doesn't degrain enough for my taste even at 3. Nice to see LSFmod added to it I will try it out aswell.

DTL
7th December 2021, 10:34
Fastdegrain(degrain=3) 0.0222x speed and gives good denoise and most noise is removed.
Fastdegrain_svg(degrain=3) 0.0245x speed but too little noise removed in comparison to the mvtools only fastdegrain..

I think GPU-based MAnalyse should be >10 times faster in compare with CPU version. But this speed numbers shows it is about equal ? What is the frame size and CPU that run at so low speed for the very small tr=3 ?

HH,
ThSAD looks good.
TrueMotion, even if false then should maybe set global=true, almost always better. [Internally Global=truemotion also switches it off when false{global = setting for motion, track whole frame panning}].
mt=MT, think Pinterf might have commented on using MT setting, saying just makes things slower doing Prefetch in AVS+, and again internally in mvtools [default should maybe not by true, anyways].

Avstp mt typically slower on new CPUs. Strange but true. Also may cause hanging. The worst is enabling both avstp and avs mt. So in the script with good servicing of user it is good to detect if avs mt is used and emit warning about disabling avstp mt for mvtools. The only reason to use avstp mt - if system is very low on RAM and can not fit avs mt without swapping. Avstp mt uses same amount of RAM as single thread. And avs mt creates lots of copies of framebuffer for each thread.

About global predictor:

I do not see if global predictor switched in PlaneOfBlocks.cpp - https://github.com/pinterf/mvtools/blob/d8bdff7e02c15a28dcc6e9ef2ebeaa9d16cc1f56/Sources/PlaneOfBlocks.cpp#L950 (at least at version 2.7.45). It is checked in all processing settings.
I found it useful and possibly developers forgot to remove 'global' param ? It is now used in 'optPredictorType=1' reduced predictors combination for better speed in 'work-in-progress' for next version.

About lambda and pnew:

lambda looks like used in MotionDistortion checking of vector candidate before checking (time consuming). So too high lambda may cause a bit better speed but lost some of better sad-wise vectors and lower degrain quality.

pnew is a additional cost penalty for new already found better-sad vector and its acting for degraining work is strange. It may degrade degraining quality with not visible benefit. So it is better to experiment with lowering pnew and lambda values depending on content.

As I see MVtools have different tasks like motion interpolation and degraining. And possibly 'truemotion' and other params defaults may be tuned for average quality of different tasks but may be not best for degraining work only.

StainlessS
7th December 2021, 13:38
So in the script with good servicing of user it is good to detect if avs mt is used and emit warning about disabling avstp mt for mvtools.
Is there some method of detecting if Prefetch is being used ?

Most of my ramblings about Mvtools is totally based on guesswork, I dont understand the code at all [I aint a CPP progger - and my math is terrible].

hello_hello
7th December 2021, 14:13
I do not see if global predictor switched in PlaneOfBlocks.cpp - https://github.com/pinterf/mvtools/blob/d8bdff7e02c15a28dcc6e9ef2ebeaa9d16cc1f56/Sources/PlaneOfBlocks.cpp#L950 (at least at version 2.7.45). It is checked in all processing settings.
I found it useful and possibly developers forgot to remove 'global' param ? It is now used in 'optPredictorType=1' reduced predictors combination for better speed in 'work-in-progress' for next version.

Are you saying it's permanently enabled? I'm certain there was a difference between global=true/false when I was playing around yesterday. Only a slight difference, but I'm sure there was one.

anton_foy,
I just noticed a difference between my script and the version of FastDegrain on the wiki in respect to LSFMod sharpening, and I don't remember if it was deliberate or just a silly. It's probably the latter but I rarely use FastDegrain with sharpening enabled so I can't remember. Anyway....
The version on the wiki sharpens the video before MSuper. Mine doesn't. I'm not sure how much difference it'd make or if one way is better than the other, but lines 75-79 in the version I posted are this (or they were, as I'll edit my previous post to change it to match the original script):

Input = !Sharp || !SFirst ? Source : \
Source.LSFMod(strength=Str, ss_x=SS, ss_y=SS, soft=Sft, overshoot=OShoot)

SuperClip = Source.MSuper(pel=Pel, mt=MT)

To match the original script it should be this:

Input = !Sharp || !SFirst ? Source : \
Source.LSFMod(strength=Str, ss_x=SS, ss_y=SS, soft=Sft, overshoot=OShoot)

SuperClip = Input.MSuper(pel=Pel, mt=MT)

Of course that's only if SFirst=true, which isn't the default.
The default is for LFSMod to sharpen the denoised clip as it would if it followed FastDegrain in a script. ie
FastDegrain()
LSFMod()

Oh, and to be pedantic, I changed the first two lines of the Output section too.

This:
Output = \
(Degrain < 1) ? Source : \

To this:
Output = \
(Degrain < 1) ? Input : \

So for SFirst=true the video will still be sharpened even if Degrain=0 :)

hello_hello
7th December 2021, 15:00
anton_foy,
Something else you might care to try. I was going to add MDegrain=4/5/6 to FastDegrain, but as I've never known if MDegrainN produces the same result as the other MDegrains I compared them today, and it appears when all else is equal, it does. So for a greater temporal range I used this script instead, It's called MDegrainNL (MDegrain Light) as the defaults do fairly light degraining, but it's just the defaults that are a little different, and MDegrainNL doesn't include sharpening.
These should produce the same result.

MDegrainNL(TR=3, thSAD=400, BLKSize=16, Overlap=4, Pel=2)
FastDegrain(Degrain=3, thSAD=400, BLKSize=16, Overlap=4, Pel=2)

Edit: By the way, when you said FastDegrain gives you 0.0222x speed is that correct, because for a 24fps source I'd interpret that to mean something like 0.53fps, which is painfully slow.

Edit: Removed the default thSAD2 value from the MDegrainNL function. See post #46.

# ===============================================================================
# ===============================================================================
# MDegrainNL (MDegrainN Light) 2021-12-08
# ===============================================================================
# ===============================================================================

function MDegrainNL(clip Source, \
int "TR", int "thSAD", int "thSAD2", int "BLKSize", int "Overlap", int "Pel", bool "Precise", \
bool "TrueM", bool "GlobalM", int "lambda", int "pnew", bool "MT", bool "LSB") {

TR = default(TR, 1) # Temporal radius
thSAD = default(thSAD, 150) # Denoising strength
BLKSize = default(BLKSize, 16) # Block size
Overlap = default(Overlap, BLKSize/2) # Block overlap
Pel = default(Pel, 2) # MVAnalyse pel (1, 2, 4)
Precise = default(Precise, false) # Recalculate
TrueM = default(TrueM, false) # Enable/disable truemotion
GlobalM = default(GlobalM, true) # Enable/disable global motion
MT = default(MT, false) # Internal multithreading
LSB = default(LSB, false) # Stacked 16-bit

# Uncommenting the two lines below results in the same MVTools2 settings as QTGMC when truemotion=false
# lambda = default(lambda, (TrueM ? 1000 : 100 ) * BLKSize * BLKSize / 64)
# pnew = default(pnew, TrueM ? 50 : 25)

Super = MSuper(Source, pel=Pel, mt=MT)

MultiVec1 = MAnalyse(Super, multi=true, delta=TR, blksize=BLKSize, overlap=Overlap, \
truemotion=TrueM, global=GlobalM, lambda=lambda, pnew=pnew, mt=MT)

MultiVec2 = !Precise ? MultiVec1 : \
MRecalculate(Super, MultiVec1, tr=TR, blksize=BLKSize/2, overlap=Overlap/2, \
thSAD=thSAD2, truemotion=TrueM, lambda=lambda, pnew=pnew)

MDegrainN(Source, Super, MultiVec2, TR, thSAD=thSAD, thSAD2=thSAD2, mt=MT, lsb=LSB) }

# ===============================================================================
# ===============================================================================

DTL
7th December 2021, 16:06
"Is there some method of detecting if Prefetch is being used ?"

It is a good question to AVS developers - may be exist or can be simply added some 'property' like _num_avs_threads. If it > 1 - the internal avstp-based mt in mvtools may be disabled. Or may be adjusted to num_cores - num_avs_threads using avstp_set_threads().

" it's permanently enabled? "

Hmm - it looks like forgotten place to optimize. If enabled it start process of estimating global motion vector (more or less time consuming). But if disabled - the motion vector still checked at the search process (may be with some default data like zero). If users frequently do not use global motion analysis - it can be disabled in processing and speed up things a bit. But it will be +1 branching in the frequently used part of program and possible +slight degradation of speed for users of global motion.
It looks it was expected global motion predictor will be most of time used. So its disabling 'global' may be sort of user-side debug option. Though it is auto-disabled with disabling of 'truemotion' group of settings. But 'truemotion' is enabled in default settings.

" I've never known if MDegrainN produces the same result as the other MDegrains"

In latest official 2.7.45 MDegrainN fallback to (faster (possibly)) MDegrainX if thSAD=thSAD2. In later testbuilds MDegrainN have additional wpow param and pinterf wrote it need to be propagated to MDegrainXs for compatibility. But it still delayed.
The speed of MDegrainXs may be a bit better in compare with MDegrainN with same tr.
Do SVP GPU-based MAnalyse can support 'large' tr like up to 128 (current MDegrainN max) ?

"thSAD2 = default(thSAD2, thSAD/2)"

It is strange to see such great limit of thSAD2 param. It will mostly waste CPU cycles for 'useless' frames at the end of 'tr'. The main idea of thSAD2 is looks like to lower some visible bugs (errors in averaging too different blocks and so blurring) with large enough tr values. If these bugs ever happens at all. It may be if thSAD is set too high for current content (its noise level).
So if thSAD2 = thSAD/2 it for MDegrain3 mean about only MDegrain2 or MDegrain1 is actually used for denoising because at the tr=3 the thSAD falls to really thSAD/2 and below noise-deviated SAD so that blocks will get 0 averaging weight in MDegrainN.
Typical default for enabling MDegrainN processing is thSAD2 = thSAD-1 .

With precise adjustment (bewteen detail blurring and noise decreasing) the thSAD value is just small above 'noise level' and there is almost no space left to lower thSAD2. So thSAD2 may be 75..90% of thSAD or just equal.

In best case 'degrain script' must not have any thSAD default (or set it to extra low like 100 or 50) so user must adjust it (increase) to its content to set and understand the relation between degraining strength and low contrast details lost. For the current user's use case.

anton_foy
7th December 2021, 21:01
@DTL
Yes I guess it is something weird with my script or my computer? The footage I use is in UHD.

@hello_hello
I use UHD-footage from my Sony A6300 camera.
My BASIQ-script is almost as fast as using only fastdegrain with mp_pipeline
and very effective (the dancing noise in my footage gives a wobbly appearance with spatial
and most temporal denoisers aswell but not with fastdegrain/mdegrain + my neo_vaguedenoiser-mocomped)

Comp:
Nvidia GeForce GTX 970
Intel Core i5-3570K 3.40GHz
32GB RAM

Script:

MP_Pipeline("""

SetFilterMTMode("DEFAULT_MT_MODE", 2)

#Index
LSMASHVideoSource("F:...mp4")

# ### prefetch: 32, 16
### ###
### platform: Win32

SetFilterMTMode("LSmashVideoSource", MT_SERIALIZED)

convertbits(16)
Deblock_QED_mt(quant1=26, aOff1=1, aOff2=1, bOff1=2, bOff2=2, uv=3)
Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false)

### platform: win32
### branch 4
### ###

### platform: Win32

### ###

# DENOISE #
#basiq(preset="high")
Fastdegrain(degrain=3)
#Fastdegrain_svp(degrain=3)

### ###
### platform: Win32

neo_f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, mt=true)

### platform: Win32

### ###

""")
trim(40,60)

#Chroma Upscaling
ConverttoYUV444()
ConvertBits(bits=10, dither=1)
Prefetch(4)

DTL
7th December 2021, 21:20
"it is something weird with my script or my computer?"

I do not know what mean Deblock_QED_mt and neo_f3kdb and what is their speed. How speed increases if disable Fastdegrain ? Also it is better not leave thSAD param to default - it should be adjusted to the current footage noise level. If there is a combined footage with different 'ISO' settings and different noise levels - it is better to process each part with different thSAD value. If your camera do not use auto-ISO at time of shooting I think it is possible to make a table of thSAD values for each ISO setting (that mean for some fixed noise levels from that hardware). And assign the required thSAD value based in the ISO setting from metadata. Or at least select it as starting value for fine tuning.

hello_hello
8th December 2021, 07:33
"thSAD2 = default(thSAD2, thSAD/2)"

It is strange to see such great limit of thSAD2 param. It will mostly waste CPU cycles for 'useless' frames at the end of 'tr'. The main idea of thSAD2 is looks like to lower some visible bugs (errors in averaging too different blocks and so blurring) with large enough tr values. If these bugs ever happens at all. It may be if thSAD is set too high for current content (its noise level).
So if thSAD2 = thSAD/2 it for MDegrain3 mean about only MDegrain2 or MDegrain1 is actually used for denoising because at the tr=3 the thSAD falls to really thSAD/2 and below noise-deviated SAD so that blocks will get 0 averaging weight in MDegrainN.
Typical default for enabling MDegrainN processing is thSAD2 = thSAD-1 .

With precise adjustment (bewteen detail blurring and noise decreasing) the thSAD value is just small above 'noise level' and there is almost no space left to lower thSAD2. So thSAD2 may be 75..90% of thSAD or just equal.

I'm not sure why I decided on
thSAD2 = default(thSAD2, thSAD/2)
but as you made me look I repeated one of yesterday's encodes with thSAD2=thSAD, and there's differences, both good and bad.

Yesterday's encode came in at 4169.06 kb/s, and after running it again today with thSAD2=thSAD, the bitrate dropped to 3679.27 kb/s, so yes obviously more noise was removed. The source is very noisy animation so I'm using:

MDegrainNL(Tr=6, thSAD=600, Pel=4)

For animation, making thSAD2 the same as thSAD caused some lines to be blurred a tad more or reduced in size where there's movement. Not a lot, but it was there.
That additional blurring increased a little when the video was encoded (compared to looking at the script output) but I assume that's because the extra noise also caused the encoder to keep a bit of extra detail too.

So after a few cigarettes and some time contemplating my naval, as I can only see differences where there's motion (in respect to blurring), and even then, only when I compare individual frames, I think I'll just remove the thSAD2 line from the script.

Thanks for that.

The speed of MDegrainXs may be a bit better in compare with MDegrainN with same tr.

When I was comparing them yesterday, degraining a 720p source in 8 bit, MDegrainN with TR=3 was 0.02fps slower than MDegrain3. Not enough to care about at that resolution, but I can't remember if I was using MT=true at the time.

In best case 'degrain script' must not have any thSAD default (or set it to extra low like 100 or 50) so user must adjust it (increase) to its content to set and understand the relation between degraining strength and low contrast details lost. For the current user's use case.

I created the MDegrainNL function for myself, and at the time it was intended as a go to function for fairly light degraining of HD video, so I added appropriate defaults to avoid typing argument names unless it's necessary. :)

hello_hello
8th December 2021, 07:55
@DTL
Yes I guess it is something weird with my script or my computer? The footage I use is in UHD.

Processing UHD is 16 bit is bound to be slow. I wouldn't go near it with my old computer. How hard does your CPU work when you're using that script?

DTL
8th December 2021, 08:46
To make 'FastDegrain' faster at 16bit inputs it may be added converting input to 8bit for MAnalyse. It should be faster and new testbuilds have only 8bit MAnalyse SIMD accelerated functions for 8x8 and 16x16 blocks only.

anton_foy
8th December 2021, 09:25
To make 'FastDegrain' faster at 16bit inputs it may be added converting input to 8bit for MAnalyse. It should be faster and new testbuilds have only 8bit MAnalyse SIMD accelerated functions for 8x8 and 16x16 blocks only.

Like this?
# Create our Super clip for MV search. Then, search for motion vectors.
super = src.MSuper(pel=pel)
super8 = src.convertbits(8).MSuper(pel=pel)
bvec3 = (degrain>=3) ? super8.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP()
bvec2 = (degrain>=2) ? super8.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP()
bvec1 = (degrain>=1) ? super8.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP()
fvec1 = (degrain>=1) ? super8.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP()
fvec2 = (degrain>=2) ? super8.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP()
fvec3 = (degrain>=3) ? super8.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, thSAD=thSAD, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=limit) : src

return src
}

How hard does your CPU work when you're using that script?

I must check it out and report back to you. I pipe it to ffmpeg and make prores 10-bit files if maybe that slows things down aswell?

anton_foy
8th December 2021, 10:54
Like this?
# Create our Super clip for MV search. Then, search for motion vectors.
super = src.MSuper(pel=pel)
super8 = src.convertbits(8).MSuper(pel=pel)
bvec3 = (degrain>=3) ? super8.MAnalyse(isb=true, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP()
bvec2 = (degrain>=2) ? super8.MAnalyse(isb=true, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP()
bvec1 = (degrain>=1) ? super8.MAnalyse(isb=true, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP()
fvec1 = (degrain>=1) ? super8.MAnalyse(isb=false, delta=1, blksize=blksize, overlap=overlap, mt=MT) : NOP()
fvec2 = (degrain>=2) ? super8.MAnalyse(isb=false, delta=2, blksize=blksize, overlap=overlap, mt=MT) : NOP()
fvec3 = (degrain>=3) ? super8.MAnalyse(isb=false, delta=3, blksize=blksize, overlap=overlap, mt=MT) : NOP()

# Degraining the video using MVDegrain. Nothing special here.
src = (degrain==1) ? src.MDegrain1(super, bvec1, fvec1, thSAD=thSAD, limit=limit) : src
src = (degrain==2) ? src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, thSAD=thSAD, limit=limit) : src
src = (degrain>=3) ? src.MDegrain3(super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, thSAD=thSAD, limit=limit) : src

return src
}



I must check it out and report back to you. I pipe it to ffmpeg and make prores 10-bit files if maybe that slows things down aswell?

Edit: I also want to mention that if I use blocksize 16 instead of 8 I get more motion blur added in moving parts even if it is moving just slightly.
Is that mainly the trade off for higher blocksizes or can another parameter compensate it?