MVTools-pfmod

[StainlessS]

Hi there.
I was going to perform temporal degrain as usual when I noticed something weird:


source=FFMpegSource2("Cricket.mov", atrack=-1)
soccer=trim(source, 0, 1534)
soccer_fields=AssumeTFF(soccer)
soccer_bob=Bob(soccer_fields)
soccer_cropped=Crop(soccer_bob, 64, 18, -32, -16)
soccer_super=MSuper(soccer_cropped, pel=2, sharp=1) # MSuper aint got no clip, and no Last defined !!!
soccer_bv1=MAnalyse(soccer_super, isb = true, delta = 1, overlap=4)
soccer_fv1=MAnalyse(soccer_super, isb = false, delta = 1, overlap=4)
soccer_bv2=MAnalyse(soccer_super, isb = true, delta = 2, overlap=4)
soccer_fv2=MAnalyse(soccer_super, isb = false, delta = 2, overlap=4)
soccer_degrain=MDegrain2(soccer_super, soccer_bv1, soccer_fv1, soccer_bv2, soccer_fv2,thSADC=200, thSAD=200)
soccer_SD=DebilinearResizeMT(soccer_degrain, 720, 576)


This doesn't work as MSuper doesn't recognize "soccer_cropped" as "clip", therefore MDegrain2 at the end complains about an invalid input.
On the other hand, this works:


source=FFMpegSource2("Cricket.mov", atrack=-1)
soccer=trim(source, 0, 1534)
soccer_fields=AssumeTFF(soccer)
soccer_bob=Bob(soccer_fields)
Crop(soccer_bob, 64, 18, -32, -16) # Assigns to implicit Last, so below MSuper gets from implicit Last and works OK.
soccer_super=MSuper(pel=2, sharp=1)
soccer_bv1=MAnalyse(soccer_super, isb = true, delta = 1, overlap=4)
soccer_fv1=MAnalyse(soccer_super, isb = false, delta = 1, overlap=4)
soccer_bv2=MAnalyse(soccer_super, isb = true, delta = 2, overlap=4)
soccer_fv2=MAnalyse(soccer_super, isb = false, delta = 2, overlap=4)
soccer_degrain=MDegrain2(soccer_super, soccer_bv1, soccer_fv1, soccer_bv2, soccer_fv2,thSADC=200, thSAD=200)
soccer_SD=DebilinearResizeMT(soccer_degrain, 720, 576)


Is it intended?
I mean, the first parameter of MSuper is supposed to be clip, right? Why doesn't it accept my custom name?


1st code block, you dont give a clip to MSuper, and Last is undefined.
2nd one, works because crop supplies implicit last with a clip, so it works, your script was at fault.

[FranceBB]

1st code block, you dont give a clip to MSuper, and Last is undefined.

But I gave "soccer_cropped" as first argument to MSuper which is the output from "Crop" and of course, since it's the very first argument, I expect it to be used as a clip. Why isn't it used as a clip?

soccer_super=MSuper(soccer_cropped, pel=2, sharp=1)

[StainlessS]

Oops sorry, I was looking for soccer_cropped.MSuper() not Msuper(soccer_cropped)
NOT your script fault. :( Senility, you gotta luv it, always good for a laugh.

Wonkey is correct, funny happenings of late.

[FranceBB]

Senility, you gotta luv it, always good for a laugh.

Nah, not your fault, the script is a bit messy to read and it may seem "stupid" given that I could have left everything as it was, but it's actually part of a bigger, more complicated script, hence the need to assign those names... I've basically got a series of events in the same tape, but whoever dubbed the thing through a VTR at the time clearly didn't care about things like inactive lines and reverse fields and levels and many other things. Heck, in some points I even had to make a mask by hand to make sure the pitch looked almost the same from every camera... (and I don't have the cameras, I had to TRIM every single part... It was a hell of a job, but apparently my boss likes that game particularly... I don't know why, though, as his favorite team lost in the Champion's League final, so to me it would be something to forget, not remember xD I shouldn't comment any further, though, as he might be lurking on Doom9, especially since he knows that I spend much time here...)


Anyway, the thing is that David is right, there have been some weird things going on in the latest AVS+ version that affect a series of filters on the "last" argument, so it's not bonded to MVTools...

(Side Note: by "David" I mean Wonkey Monkey, although I never understood why he changed his nickname a while ago xD)

[feisty2]

@pinterf

there's something fishy with MRecalculate, it seems we need to scale up searchparam when refining vectors to a smaller block size to get the correct result.
theoretically, this

vec = sup.MAnalyze(blksize=64, overlap=32, search=3, searchparam=2, badrange=-24)
vec = sup.MRecalculate(vec, blksize=2, overlap=1, search=3, searchparam=2, thsad=0)
MDegrainN(clp, sup, vec, thsad=2000)

should be identical to simply

vec = sup.MAnalyze(blksize=2, overlap=1, search=3, searchparam=2, badrange=-24)
MDegrainN(clp, sup, vec, thsad=2000)

but the results are far from being identical, the one using MRecalculate has A LOT of residual ghosting artifacts from blksize=64. To produce something similar to the one without MRecalculate (I'm not sure if it's bit-identical tho), you have to scale searchparam up to 64 for MRecalculate

vec = sup.MAnalyze(blksize=64, overlap=32, search=3, searchparam=2, badrange=-24)
vec = sup.MRecalculate(vec, blksize=2, overlap=1, search=3, searchparam=64, thsad=0)
MDegrainN(clp, sup, vec, thsad=2000)


have you checked if this is a bug or something?

[Boulder]

I posted an issue to the MVTools2 Github tracker, but I'll post it here as well in case someone can verify my findings.

This is a rather strange issue and I don't know if MVTools, Avisynth+, AVS2YUV or x265 is the real culprit.. nevertheless, I was able to determine that MVTools can be used to trigger it.

Using this kind of a script (don't worry about the supermdg bit being "unneeded", I used my denoising function as a base), I get jerky movement in the beginning of the encoded clip where the Enterprise appears on the screen. This happens if Prefetch is used - if I comment it out, no problems. The defect is not consistent between encodes, for example these are two consecutive test encodes with the same script and encoder parameters:

encoded 198 frames in 37.80s (5.24 fps), 1742.24 kb/s, Avg QP:24.60
encoded 198 frames in 36.98s (5.35 fps), 1721.16 kb/s, Avg QP:24.59

dgsource("c:\x265\tng\tngjerk.dgi")
convertbits(16)
superanalyse = msuper(pel=2, sharp=2, rfilter=4, chroma=true)
supermdg = msuper(pel=2, levels=1, sharp=2, rfilter=4, chroma=true)
fv1 = manalyse(superanalyse, isb=false, delta=1, blksize=16, overlap=8, search=5, searchparam=16, pelsearch=8, truemotion=false)
bv1 = manalyse(superanalyse, isb=true, delta=1, blksize=16, overlap=8, search=5, searchparam=16, pelsearch=8, truemotion=false)
mdegrain1(supermdg, bv1, fv1, thsad=100, thsadc=100)
Prefetch(threads=24, frames=24)

The encoder command line is this:
c:\x265\avs2yuv64.exe -no-mt -depth 16 "c:\x265\tng\tngjerk.avs" - | c:\x265\x265.exe -F 2 --input - --y4m --input-depth 16 --dither --sar 1:1 --profile main10 --ctu 32 --preset slower --merange 58 --crf 19 --output "c:\x265\tng\tng.hevc"

The cache settings are these:
SetMemoryMax(20480)
SetCacheMode(1)
Cache mode 2 didn't change things, still jerky.

I just tested removing truemotion=false from the script and the defect is gone. However, in my earlier tests this was also not very consistent and seemed to depend also on the x265 parameters.

Some samples:
https://drive.google.com/file/d/1e3c3Ysp80URNSQRQRABweoD41Sao3PQo/view?usp=sharing (original)
https://drive.google.com/file/d/1OdNCGdB0gqtBEQWgmlbAW0ndDyg95ugG/view?usp=sharing (with Prefetch, jerky)
https://drive.google.com/file/d/1NpToBeb7yFBjTPKS0-XR4V6zTT0ytxQm/view?usp=sharing (without Prefetch, no issues)


EDIT: just remembered this issue: https://forum.doom9.org/showthread.php?p=1698102#post1698102
Seems that adding RequestLinear(clim=100) after DGSource fixes the issue.

[pinterf]

New MvTools 2.7.44 (https://github.com/pinterf/mvtools/releases/tag/2.7.44)

- 2.7.44 (20201214)
- MAnalyze: fix motion vector generation inconsistencies across multiple runs.
Note: when internal multithreading is used (avstp + mt=true), inconsistencies will still occur by design.

[StainlessS]

Cheers P.

[FranceBB]

x86 and x64 builds
Windows XP is still supported, special DLLs with v141_xp toolset

ain't that sweet? Thanks! :D

[takla]

Is it known that large thSAD values (even the default 400) produces artefacts?
You can use this 25 seconds lossless video to test filters on yourself (https://www.mediafire.com/file/ps0j63m2t6pgqx8/RAW.mkv/file)

I'll use frame 480 in the examples below.
Raw, no filters.
https://i.ibb.co/XCHfPLD/RAW.png

MDegrainNL (https://pastebin.com/raw/NARWiZL7)(original from hello_hello (https://forum.doom9.org/showthread.php?p=1861146#post1861146), somewhat modified by me)
No artefacts of any kind and decent denoising.
https://i.ibb.co/175QJmg/MDegrain-NL-th-SAD150.png

MDegrainNL thSAD=400 (check the characters head for black pixels, also faded outlines)
https://i.ibb.co/30ZJrk0/MDegrain-NL-th-SAD400.png

The amount of artefacts are even worse with something like MCDegrainSharp (http://forum.doom9.net/showpost.php?p=1855907&postcount=31) to the point I'd never use or recommend it to anyone

MCDegrainSharp(truemotion=false) (check for the massive artefact on the wall under the characters head)
https://i.ibb.co/16fPMGz/MCDegrain-Sharp-TM-False.png

MCDegrainSharp(truemotion=true) (less obvious artefact with truemotion enabled, but it is still there, almost the same spot, just slightly shifted to the right)
https://i.ibb.co/gdxhG4P/MCDegrain-Sharp-TM-True.png

I also used ConvertBits(16) before the filter and ConvertBits(bits=10, dither=1) after.

[real.finder]

yes it's known thing, it's all about the motion vectors will be wrong in cases like fades or dark frames like this, in your case (dark frames) you can use Dither_Luma_Rebuild to feed super clip for motion analysis to get better motion vectors

[Boulder]

It would be interesting to see how the vectors look (MShow) and if the issue can be helped by using Dither_Luma_Rebuild or MRecalculate or dct=1 or dct=5.

[takla]

MDegrainNL_v1 (16-235, old) (https://pastebin.com/2nhsZuZX)
MDegrainNL_v2 (0-255, new, more accurate) (https://pastebin.com/Wi3K7f74)

v1
https://i.ibb.co/175QJmg/MDegrain-NL-th-SAD150.png

v2
https://i.ibb.co/3yJc9xg/NEW.png

If you zoom in, and look very hard you can see the characters nose hole slightly better in the v2 version. At basically no performance penalty, this is a nice improvement. I will do some more tests with higher thSAD later.

In the meantime, if someone could give me some help on how to add MRecalculate i'd appreciate it.

[ChaosKing]

From the SMDegrain wiki http://avisynth.nl/index.php/SMDegrain

Str float (0.0-8.0, default 1.0)
Gamma correction. With this parameter you control the strength of the brightening of the prefilter clip, good for when problems with dark areas arise. Using this internally instead of externally creates less quantization artifacts, since it's done in the same stage as the TV->PC range conversion.

[takla]

MDegrainNL_v3 (https://pastebin.com/MVRCnxsr)

Added (pre)blur to suppress initial noise, for better motion vectors.

(I did play around with pre-amping gamma, which indeed got rid of the smaller black artefacts at higher thSAD. You can add gamma_y=128 to ColorYUV if you need a higher thSAD. For me though the sweetspot is 150, as increasing it to 200 didn't really change filesize but slightly dimmed the nose hole again)

Overall the v3 provided better denoising, performance and no artefacts, when compared to SMDegrain(defaults) and MCDegrainSharp(defaults) on the test clip posted above.

[takla]

One more update to this script:


function MDegrainNext(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", float "Sharpen", float "Blur")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 16)
Overlap = default(Overlap, BLKSize/2)
Sharpen = default(Sharpen, 0.2)
Blur = default(Blur, 1.0)

Super_Sharp = Input.Sharpen(Sharpen).MSuper(pel=1)
Super_Blur = Input.Blur(Blur).MSuper(pel=1)
Multi_Vector = Super_Blur.MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap)

Input.MDegrainN(Super_Sharp, Multi_Vector, TR, thSAD=thSAD, thSAD2=thSAD/2, thSADC=thSADC, thSADC2=thSADC/2)
}


Info:

MDegrainNext v1 Documentation
Original by hello_hello https://forum.doom9.org/showpost.php?p=1861146&postcount=10
Modified by Takla

Why did I mod this: Needed something easy to read that only makes use of internal functions (excluding MVTools2)
This script is intended to be set-and-forget, with a superb quality-to-speed ratio and no artefacts.

thSAD & thSADC = Luma & Chroma Noise sensitivity.
Leave as is.
Values too low can cause blockiness and too high can cause black spot artefacts.
Default is 150 & thSADC=thSAD.

thSAD2 & thSADC2 = Noise sensitivity falloff.
Leave as is.
Default is thSAD/2 & thSADC/2.

TR = Temporal Radius.
The main setting you should be tweaking.
Higher values allow for more noise to be removed, at the cost of speed.
Higher values might also require you to lower thSAD to avoid black spot artefacts.
Default is 3.

BLKSize = Block size in pixels.
Lower means slower and more accurate but not necessary visible better.
Default is 16.
Recommendation: 540p=8, 1080p=16, 2160p=32

Overlap = Modifies Block size behaviour.
Leave as is.
Use BLKSize/4 for the best quality-to-speed ratio.
Use BLKSize/2 to preserve very fine detail. Slower.
Default is BLKSize/2.

Sharpen = Increases sharpness.
Much more accurate when used with Super, instead of calling it as a standalone after MDegrainNext.
You only want to do a little bit of sharpening, as it will increase bitrate requirements.
You can always sharpen more via media player (mpv or mpc).
Good values are 0.1 to 0.5.
Default is 0.2.

Blur = (Pre)blurs the input to avoid false motion vectors.
Allows for more accurate motion vectors.
Good values are 0.5 to 1.0, or 1.58 for very noisy sources.
Default is 1.0.


And maybe change pel back to 2 for SD videos.

[tormento]

MDegrain: out16 parameter, which outputs 16 bit result from 8 bit input
Would be possible to have out10 parameter, as it is the most used bit depth with x265, and out12, for HDR, or the 16 number is given from a stacked image and so only 8 multiples can be used?

[DJATOM]

But that can be achieved by simply put ConvertBits(10/12/14) after MDegrain :)

[tormento]

But that can be achieved by simply put ConvertBits(10/12/14) after MDegrain :)
I am doing various benchmarks and out16 outperforms any other kind of script with ConvertBits. I will post results very soon.

[Boulder]

It's very odd if it does that. I would expect it to be internally like ConvertBits(16) at the end of the whole processing chain.

[pinterf]

out16 is simply omitting a final conversion of the intermediate result back to 8 bits. This internal result is always of 16 bits precision for 8 bit sources. This is the same as lsb_out works but the result is not stacked but a real 16 bit clip.

[tormento]

Please look here (https://forum.doom9.org/showthread.php?p=1935035#post1935035).
out16 is simply omitting a final conversion of the intermediate result back to 8 bits. This internal result is always of 16 bits precision for 8 bit sources. This is the same as lsb_out works but the result is not stacked but a real 16 bit clip.
So why there is so much difference between the results? 16 bit source should give the same result as out16 and 10/12 bit source should be a lot faster than out16 too.

[pinterf]

SMDegrain <> MDegrain

[tormento]

SMDegrain <> MDegrain
I understand. As I asked before, would it be possible to have 10 bit internal precision and output for 8 bit source too instead of 16 only?

[real.finder]

I understand. As I asked before, would it be possible to have 10 bit internal precision and output for 8 bit source too instead of 16 only?

start read from here https://forum.doom9.org/showthread.php?p=1835712#post1835712

[takla]

I understand. As I asked before, would it be possible to have 10 bit internal precision and output for 8 bit source too instead of 16 only?

Why not do the following with an 8 bit source:


ConvertBits(16)
MDegrain()
ConvertBits(bits=10, dither=1)


As explained here (https://forum.doom9.org/showpost.php?p=1835712&postcount=464), 10 to 16 bit makes no difference in speed and so you should either convert to 16bit or don't use out16 / ConvertBits at all.

[joka]

If I use MCompensate
s = c.MSuper(pel=2)
vb1 = s.MAnalyse(isb=false, delta=1, blksize=bs, overlap=ov)
cp1 = c.MCompensate(s, vb1)

and provide blksize and overlap I get (depending on the parameters) an unprocessed strip (with cp1 == c) on the bottom and the right border.
Further evaluation shows the same for MDeGrain (output of MDeGrain == c). With MDeGrain it is easier to compare/see. Height / width of strip depends on parameters.

Examples:
blksize/overlap
8/0 ok., nothing
8/4 ok., nothing
16/0 bottom
16/8 ok., nothing
32/0 bottom
32/8 bottom, right
32/16 bottom
64/0 bottom, right

Clip size is 1440x1080.
I use mvtools-2.7.43-with-depans20200602, AMD x64.

Do I something wrong? Is this a known issue? Exists a workaroud?

[StainlessS]

Joka, try same with 2.2.7.44:- https://github.com/pinterf/mvtools/releases
Change log
** 2.7.44 (20201214)**

MAnalyze: fix motion vector generation inconsistencies across multiple runs.
Note: when internal multithreading is used (avstp + mt=true), inconsistencies will still occur by design.


EDIT: Always a good idea to try latest version before reporting problem, if same then re-submit problem report.

[pinterf]

MDegrain is simply copying 'uncovered' bottom and right regions. Uncovered means that these pixels are not covered by whole blocks of the given blocksize.

[joka]

Good to know. I tested blksize/overlap - 24/12. This seems to solve it for clip size 1440/1080 and 1920/1080.
Thank you pinterf.

[zorr]

The current documentation on MAnalyse states about overlap: "Must be even and less than block size (up to blksize/2 for MCompensate)".

However, when I try overlap > blksize/2 I get this error message "MAnalyse: overlap must be less or equal than half block size". I don't have MCompensate in the script. MVTools x64 version 2.7.44.

So is the documentation or the code wrong? :confused:

EDIT: Seems like MAnalyse also doesn't accept negative levels.

[zorr]

The current documentation on MAnalyse states about overlap: "Must be even and less than block size (up to blksize/2 for MCompensate)".

I confirmed that this is just an error in the documentation (and it has been like that a long time). I do wonder though would it be possible to get rid of this limitation. It's usually the largest possible overlap that gives the best quality so it would be interesting to see if the trend (larger = better) would continue even with overlap > blksize/2.

EDIT: Seems like MAnalyse also doesn't accept negative levels.

This was my own error, the levels parameter was just too negative and there weren't any levels left.

[Dogway]

Would it make sense to implement a half pel option to increase speed for UHD clips or is it the same as increasing blksize and overlap?

[Boulder]

Isn't half pel the same as pel=2?

[Dogway]

I meant pel=0.5, double pel in mvtools terms

[StainlessS]

I meant pel=0.5, double pel in mvtools terms
Would that not have [a sort of] effect of lowering effective resolutuion of the result clip ?
[would seem to me that pel=1, and bigger blocksize, is the way to go]

[Dogway]

I don't think so, many prefilters use gaussian type blurs for easier motion search, lowering the resolution is just the same but with less pixels (faster). Also many UHDs are upscaled conversions.

I don't know what others have to say, seems to me that pel=2 would also equal to pel=1 and smaller blocksize?

[feisty2]

it is equivalent to performing motion estimation on a downscaled version of the video, then you 2x upscale the motion vectors

[StainlessS]

it is equivalent to performing motion estimation on a downscaled version of the video, then you 2x upscale the motion vectors

And, Would that not have [a sort of] effect of lowering effective resolution of the result clip ?

EDIT: Arh, the new avatar is a cat or something, hard to figure out what it is.

[Dogway]

it is equivalent to performing motion estimation on a downscaled version of the video, then you 2x upscale the motion vectors

Wow, thanks, already updated SMDegrain with the changes, didn't remember MScaleVect

[tormento]

Wow, thanks, already updated SMDegrain with the changes, didn't remember MScaleVect
Where is it? :)

[Dogway]

Haven't uploaded yet, it's 3 times faster but there's some bad motion vectors of some sorts that I'm debugging.

[tormento]

it's 3 times faster
WTF.

Will wait your debugged version for next encodings.

[StainlessS]

Maybe downscaled pel=2, [about] same as normal scale pel=1. [but downscale faster]

[kedautinh12]

3 times faster????? Wth??

[Dogway]

Ok, here it is RC3 (https://github.com/Dogway/Avisynth-Scripts/blob/master/SMDegrain%20v.3.2.0d/SMDegrain%20v3.2.0d.avsi), the problem was that UHD is PQ encoded so the little contrast makes values behave differently, so I recommend to decode and tonemap beforehand or at least feed one as a prefilter.
Then obviously you are getting 1080p vectors, which aren't as good as 2160p vectors, but also not worse than 1080p in any case. I tried different settings but no one made a sustainable improvement except for "limit", so for UHD is by default set to 2, play with the value.
I also continued with the cleanup, added and removed a few more things.

[tormento]

Ok, here it is RC3
Will try ASAP.
I recommend to decode and tonemap beforehand or at least feed one as a prefilter.
Mmmm.... could you implement some?

P.S: Do you plan to keep on relying on ZF_Shared?

[Dogway]

Yes, it's on my todo list for Transforms Pack 2.0, but it's not going to happen soon, in any case it's going to be CPU bound and hence slow.

I recommend you to use DGHDRtoSDR(mode="pq",white=100), it's CUDA based.

[tormento]

I recommend you to use DGHDRtoSDR
Please, if you find some time, implement different tone mapping as you did with prefilter parameter.

[Dogway]

Yes that's my plan, in the realm of what is possible with avs+ I will try to run a per-scene rolling average of peak luminance.