MVTools-pfmod

[feisty2]

Why need motion analysis in float? In most cases you cannot even separate 8 bit results from 10 or 16 bit results. Do the motion analysis fast, in 8 or 10 bits. Then apply it on the 32 bit float clip. MDegrain, MFlow, etc.. all work with float. You can even do it on RGB, while motion analysis can be done on the YUV ceonverted clip. I have a feeling that 32 bit float is not even enough for SAD, it has only 23 significant bits afaik, while mvtools is using 32 or 64 bits for summing up SADs. I think it is more interesting to examine the effect of mv precision if we put motion vectors to floating point.
And we are talking about precision? What is the base data we are working on? It's not even a linear one. Did anybody do tests on making vectors from a more linear 16 bit data converted from a 8 or 10 bit source? We do averaging, resizing, adding and subtracting and interpolating on gamma distorted videos. I think it's not the float clip-based analysis which is the bottleneck.

the internal precision of mvtools-sf is double, it even requires the double precision fftw3, not fftw3-f.

[ChaosKing]

Chaos, Was trying to find your images, and have seen links to zopti thread, perhaps in there somewhere.
Any of these images familiar:- https://www.google.com/search?source=univ&tbm=isch&q=%22chaosking%22+near+(%22mvtools%22+%26+%22smdegrain%22)&sa=X&ved=2ahUKEwjF8ZTp3InpAhXIUxUIHUo9BokQsAR6BAgHEAE&biw=1920&bih=886

Nope. But at least I learned a new img search technique :D
It was a frame from the Lupin III bluray (season 1, Anime from 1971)

[ChaosKing]

I just quickly looked over some scenes I remember had problems. They look fine now... So either the bugs were fixed over time or even worse: my ffms2 version was not frame accurate back then :devil:

EDIT
I think I have a VS portable folder with plugins and scripts from that time somewhere. Then I could check for sure.

[tormento]

I have just noticed about SVPFlow, using GPU. Is it a feature that can be ported in MVTools too?

[MeteorRain]

Is the GPU feature open source? My memory told me it's closed source.

[tormento]

Is the GPU feature open source? My memory told me it's closed source.


I didn’t mean to import it but to take the idea to develop same or similar workflow.

[DJATOM]

OpticFlow sources can be obtained via Nvidia developer site (https://developer.nvidia.com/opticalflow-sdk)

[Selur]

SVP Team did share a modified mvtools2 over at
The latest version of modified MVTools is 2.5.11.9 (28.02.2012) and it's based on official MVTools 2.5.11.3 release (19.09.2011).
The distribution includes:

mvtools2.dll library (requires VC++ 2008 SP1 run-time)
sources of modified MVTools2
libflowsse.dll and libflowgpu.dll libraries
sample script
readme and licenses


source: https://www.svp-team.com/wiki/Plugins:_MVTools2

[magnetite]

Speaking of GPU acceleration, I was curious if something like MDegrain could use it? Reason I ask is because there are other GPU based denoisers out there, such as FFT3DGPU and KNLMeansCL.

[tormento]

SVP Team did share a modified mvtools2
Nice but a bit outdated since based on MVTools 2.5.11.3.

Pinterf? MeteorRain? :)

[Reel.Deel]

SVP Team did share a modified mvtools2 over at

source: https://www.svp-team.com/wiki/Plugins:_MVTools2

The SVP version includes these changes:


MAnalyse: Can be faster than original version (with chroma=true) by 20-40%, look at PlaneOfBlocks.h for changes. Also there's one additional parameter: search_coarse ....


MFinest: This function is commented out in latest MVTools 2.5 versions and it's obviously a mistake cause it can helps a lot with caching the big frame for hpel (pel=2) and qpel (pel=4) modes.


MSmoothFps: It's a completely new function that encapsulates MFlow, MFlowFps and MBlockFps functionality and with GPU support enabled ..... Note: this part is not open source so ....


The MAnalyze change was included in the VS port in the early stages, discussion starts here: https://forum.doom9.org/showthread.php?p=1695899#post1695899

Pinterf later mades those changes as well (additional search_coarse parameter is not included): https://forum.doom9.org/showthread.php?p=1790290#post1790290

-----
Edit:
OpticFlow sources can be obtained via Nvidia developer site (https://developer.nvidia.com/opticalflow-sdk)

The SVPFlow plugin already has a function that uses this. Too bad is closed source.


SVSmoothFps_NVOF(): The same as SVSmoothFps but the motion vectors are acquired in-place via NVidia Optical Flow API so there's no need for SVSuper/SVAnalyse at all.

[tormento]

IMHO the way to go is to offload as many loads to GPU as possible. Of course this will change some paradigma. The power of modern GPUs is undeniable, more power for same bucks than CPUs.

[Emulgator]

Hi pinterf !
Many thanks for your continued work on useful plugins and Avisynth+ !
Donation time for me. HowTo ?

[pinterf]

New release, please test it.
https://github.com/pinterf/mvtools/releases/tag/2.7.42

Update after Boulder's request: limit and limitc parameters of MDegrain functions were integer and accepted values 0-255. (unlike the VapourSynth version which does not auto scale parameters)

So far for 16 bit output the minimum "1" would still mean 256 levels.

Now limit and limitc parameters are of float type.
By specifying limit=0.1*255 you can effectively limit the change to 25 for a 16bit clip.

The other big change is that functions are now copying frame properties when using Avisynth+ 3.6. Frame properties are not used however, that is the filters simply do not choke them and do not put them to dev/null :)

The Depan-family update contains only this new fp-copy feature, there is no functional change there.

- 2.7.42 (20200522)
- MDegrain family: limit and limitc to float, allowing more granurality for 10+ bit depth
- Update Avisynth headers, use V8 interface frame property copy if available

[zybex]

Hello everyone.

Dear pinterf

Thanks for the support and further development of the MVTools-pfmod filter.
But unfortunately, I found one annoying and unpleasant bug that I can’t put up with, because I'm a perfectionist.

All versions of this filter (x86 and x64) that are newer than the original latest version v2.5.11.22 from Fizick contain this bug:
for "MDegrain2" command:
The coding result is different every time.

I checked all of these versions:
2.6.0.5 / 2.7.0.22c / 2.7.0.1 / 2.7.34 / 2.7.39 / 2.7.40 / 2.7.41 / 2.7.42
this bug is present in all of them

The differences are in the heap of different pixels in the heap of different frames, and as a rule, starting from about the middle of the video with a length of 3 minutes. Visually, these differences are almost imperceptible, but they are, and there are a lot of them, and this I think affects the quality of compression of MPEG4 codecs.
Moreover, there are more differences if you encode in AVI XviD VFW and less if you encode in AVI x264-Lossless VFW.

How to check this problem?
You make 3 attempts to encode the same AVS file in a row, you get 3 different AVI files. As a rule, their length will be slightly different, already by this sign it can be judged that there are differences in the frames. But even if in two of the three AVI files the length is the same, then anyway, by bitwise comparing them (fc /b file1.avi file2.avi >log), it turns out that they have differences.
If necessary, I can later lay out instructions on how to find exactly which frames differ in different AVI video files through AVISynth, then you can save them as pictures and compare them visually through "Beyond Compare 3".

Here are the contents of my AVS script:
(I just simplified the script as much as possible for the test so that other commands and filters did not affect the test)

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\mvtools2.dll")
mpeg2source("Michael Gray - The Weekend.d2v")
super = MSuper (hpad=8, vpad=8, pel=4, levels=0, chroma=true, sharp=2, rfilter=1)
backward_vec1 = MAnalyse(super, blksize=16, search=5, isb=true, delta=1, overlap=8)
backward_vec2 = MAnalyse(super, blksize=16, search=5, isb=true, delta=2, overlap=8)
forward_vec1 = MAnalyse(super, blksize=16, search=5, isb=false, delta=1, overlap=8)
forward_vec2 = MAnalyse(super, blksize=16, search=5, isb=false, delta=2, overlap=8)
MDegrain2(super, backward_vec1, forward_vec1, backward_vec2, forward_vec2, thSAD=200, plane=0)

Here is my source (DVD VOB music video)
http://worldclips.ru/clips/Michael+Gray/The+Weekend+%28feat+Shena%29
(you need to log in to download it)

DGDecode.dll - v1.5.8.0 x86
http://rationalqm.us/dgmpgdec/dgmpgdec158.zip
DGDecode.dll - v1.5.8.0 x64
http://www.mediafire.com/?c0wmemj5jam

the original latest version v2.5.11.22 from Fizick (good, no bugs)
www.avisynth.nl/users/fizick/mvtools/mvtools-v2.5.11.22.zip

I use VirtualDub2, but the result does not depend on the version of VirtualDub.

Please try to fix this bug, and achieve a stable encoding result using the MVTools-pfmod filter so that three different encoding attempts give the same encoding result bit by bit (for "XviD 1.3.7 VFW" and "x264-Lossless VFW").

P.S.
I would reconcile myself to using this slower but more stable version v2.5.11.22 from Fizick, but I decided to switch to AVISynth+ and use x64 versions of filters and multi-threading, but unfortunately v2.5.11.22 does not have x64 version.

[feisty2]

the problem does not exist in vaporsynth mvtools, there must be some uninitialized memory bug left unfixed in the avs branch.

[pinterf]

the problem does not exist in vaporsynth mvtools, there must be some uninitialized memory bug left unfixed in the avs branch.
Yep, seems a nasty one, if it happens with 2.6.0.5.
And I'm so happy since I was able to read the site's warning and I could refresh my knowledge in Russian I have started learning in school for some 40 years ago :)

[pinterf]

But unfortunately, I found one annoying and unpleasant bug that I can’t put up with, because I'm a perfectionist.

Created an issue on my git repo

[StainlessS]

@ Zybex,
Not many posts since you joined in 2007, this one a gem :goodpost:
Thanks.

[pinterf]

@Zybex: Could not reproduce. 8 out of 8 sessions are bit identical. Do you have accidentally an avstp.dll in your plugins directory?
Edit: yes. avstp.dll provides internal multithreading for specific Mvtools2 filters. Such filters have a bool "mt" parameter, default=true. (see in html docs)
So if avstp.dll exists, all such functions are working with internal multithreading.

This feature is part of mvtools since 2.6.0.5. And because this version was my base version, all versions I released later have inherited its behaviour.

In your specific case, choose one
- remove avstp.dll
- add an mt=false for MAnalyze (this is the culprit)
- wait until I or someone finds the bug

[zybex]

@Zybex: Could not reproduce. 8 out of 8 sessions are bit identical. Do you have accidentally an avstp.dll in your plugins directory?

Yes, i have it here:

c:\Program Files (x86)\AviSynth+\plugins64\avstp.dll
c:\Program Files (x86)\AviSynth\plugins\avstp.dll

Is it wrong ?

[real.finder]

Yes, i have it here:

c:\Program Files (x86)\AviSynth+\plugins64\avstp.dll
c:\Program Files (x86)\AviSynth\plugins\avstp.dll

Is it wrong ?

try after remove it

[pinterf]

Yes, i have it here:

c:\Program Files (x86)\AviSynth+\plugins64\avstp.dll
c:\Program Files (x86)\AviSynth\plugins\avstp.dll

Is it wrong ?
Not wrong, but for MAnalyze something is buggy.

[zybex]

@Zybex:
In your specific case, choose one
- remove avstp.dll
- add an mt=false for MAnalyze (this is the culprit)

@pinterf

Thank you for the quick and correct hint of ways to solve the problem of instability of the encoding result in MPEG-4. It really turned out that way.

The most correct method to solve the problem:
- add multithreading disable to the "MAnalyze" function by adding the command "mt=false"

So my script turned out like this:

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\mvtools2.dll")
mpeg2source("Michael Gray - The Weekend.d2v")
super = MSuper (hpad=8, vpad=8, pel=4, levels=0, chroma=true, sharp=2, rfilter=1)
backward_vec1 = MAnalyse(super, blksize=16, search=5, isb=true, delta=1, overlap=8, mt=false)
backward_vec2 = MAnalyse(super, blksize=16, search=5, isb=true, delta=2, overlap=8, mt=false)
forward_vec1 = MAnalyse(super, blksize=16, search=5, isb=false, delta=1, overlap=8, mt=false)
forward_vec2 = MAnalyse(super, blksize=16, search=5, isb=false, delta=2, overlap=8, mt=false)
MDegrain2(super, backward_vec1, forward_vec1, backward_vec2, forward_vec2, thSAD=200, plane=0)

And this completely solved the problem of instability of the encoding result, for "MVTools-pfmod v2.7.42" x86 and x64.

But at the same time, the CPU load became 2 times less, and the processing time now became 2 times longer.

In the same way, the option to delete the file "avstp.dll" solved the problem, but this is the wrong solution to the problem, because this will disable multithreading in all filters that support it.

I think it’s worth adding a comment to the instruction for the "MVTools-pfmod" filter regarding the need to add "mt=false" to "MAnalyze"

I don’t know why this problem manifests so far only with me.

Just in case, my PC configuration:
CPU: Intel Core i5-6500 3.20Ghz
RAM: 16Gb
OS: Windows 10 Corporate LTSC x64 10.0.17763.1131

As I understand it, the version "v2.5.11.22 from Fizick" does not have this problem because it initially does not support multithreading for the "MAnalyze" function at all and does not have the command "mt=" .

Nevertheless, it would be good to fix and achieve stable operation of the "MVTools-pfmod" filter without having to disable multithreading in MAnalyze, because using this filter without multithreading slows it down by 2 times - this is very disappointing.

P.S.
after adding commands to the script:
SetFilterMTMode ("DEFAULT_MT_MODE", 2) # at the beginning of the script
Prefetch (4) # at the end of the script
processing speed has increased significantly and it suits me

[DJATOM]

There's actually avs+ multithreading available for the task, so just ditch avstp and add prefetch(4) in the end. It will be even faster

[StainlessS]

Nevertheless, it would be good to fix and achieve stable operation of the "MVTools-pfmod" filter without having to disable multithreading in MAnalyze, because using this filter without multithreading slows it down by 2 times - this is very disappointing.

Not wrong, but for MAnalyze something is buggy.

I think Pinterf wanted you to remove the avstp.dll or disable MT to see if it did indeed avoid the problem, so as to be sure that mt in MAnalyze is the culprit, it is I think not suggested that this is a permanent fix.

[ravewulf]

Make sure you have the AviSynth+ build from https://github.com/AviSynth/AviSynthPlus/releases then use the modified script below.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\mvtools2.dll")
mpeg2source("Michael Gray - The Weekend.d2v")
super = MSuper (hpad=8, vpad=8, pel=4, levels=0, chroma=true, sharp=2, rfilter=1)
backward_vec1 = MAnalyse(super, blksize=16, search=5, isb=true, delta=1, overlap=8, mt=false)
backward_vec2 = MAnalyse(super, blksize=16, search=5, isb=true, delta=2, overlap=8, mt=false)
forward_vec1 = MAnalyse(super, blksize=16, search=5, isb=false, delta=1, overlap=8, mt=false)
forward_vec2 = MAnalyse(super, blksize=16, search=5, isb=false, delta=2, overlap=8, mt=false)
MDegrain2(super, backward_vec1, forward_vec1, backward_vec2, forward_vec2, thSAD=200, plane=0)
Prefetch(4) #Controls how many threads to use

Try playing around with the number of threads to maximize the performance. On my own systems, 1.5x the number of physical cores seems to work well

Athlon II X4 640 (4 cores) = Prefetch(6)
Ryzen 7 1700 (8 cores/16 threads)= Prefetch(12)
Ryzen 9 3900X (12 cores/24 threads) = Prefetch(18)

[zybex]

Try playing around with the number of threads to maximize the performance.

yes, after adding commands to the script:
SetFilterMTMode ("DEFAULT_MT_MODE", 2) # at the beginning of the script
Prefetch (4) # at the end of the script
processing speed has increased significantly and it suits me.

thank you for reminding.

[real.finder]

bug in mcompensate when using grayscale

colorbars()
converttoy8
clp=last
super = clp.MSuper(pel=2, sharp=1)

bv1 = manalyse(super,isb=true, truemotion=false, delta=1,blksize=16,overlap=8)
fv1 = manalyse(super,Isb=false, truemotion=false, delta=1,blksize=16,overlap=8)

bc1 = mcompensate(clp, super,bv1)
fc1 = mcompensate(clp, super,fv1)

interleave(fc1, clp, bc1)

it crash

[Groucho2004]

it crash
I can reproduce that:
Exception 0xC0000005 [STATUS_ACCESS_VIOLATION]
Module: C:\Windows\system32\VCRUNTIME140.dll
Address: 0x000007FEEB9E1749

[pinterf]

New build with MCompensate greyscale+overlaps issue fix:
MvTools2 2.7.43 + depans (https://github.com/pinterf/mvtools/releases/tag/2.7.43)

[real.finder]

New build with MCompensate greyscale+overlaps issue fix:
MvTools2 2.7.43 + depans (https://github.com/pinterf/mvtools/releases/tag/2.7.43)

yes, this fix greyscale issue :thanks:

[Boulder]

Would something like this be close to creating vectors with hierachical motion estimation? I was thinking of testing it with UHD sources, the old methods may have difficulties in fast motion with framesize being so huge.

super_quarter = clip.Spline36Resize(width=clip.width/4, height=clip.height/4).MSuper(pel=1)
super_half = clip.Spline36Resize(width=clip.width/2, height=clip.height/2).MSuper(pel=1)
super_full = clip.MSuper(pel=1)

fv1 = super_quarter.MAnalyse(isb = false, blksize=16, overlap=8)
bv1 = super_quarter.MAnalyse(isb = true, blksize=16, overlap=8)
fv1 = MScaleVect(fv1, 2)
bv1 = MScaleVect(bv1, 2)
fv1 = super_half.MRecalculate(fv1, blksize=16, overlap=8)
bv1 = super_half.MRecalculate(bv1, blksize=16, overlap=8)
fv1 = MScaleVect(fv1, 2)
bv1 = MScaleVect(bv1, 2)
fv1 = super_full.MRecalculate(fv1, blksize=16, overlap=8)
bv1 = super_full.MRecalculate(bv1, blksize=16, overlap=8)


http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/AV0405/ZAMPOGLU/Hierarchicalestimation.html

[feisty2]

how is it any different from gradually decreasing blksize on the same super clip, keeping the blksize fixed while upsizing the super clip is essentially the same as keeping the super clip fixed while decreasing the blksize, they cover the same percentage of the image area

[feisty2]

except the quality would be much better if all hierarchies of motion estimation were conducted at the finest level, larger blksize on the finest super clip is much more accurate than smaller blksize on a reduced super clip, even tho both methods have the same receptive field.

[Boulder]

Isn't the radius of searching essential in this case? In the quarter and half sized super clip, it covers a larger area around the block.
I did test that approach briefly and it produced worse vectors than simply reducing the blocksize with recalculating. It also found movement quite deep inside the black borders (movie Interstellar, which has them IMAX parts to fill the whole frame at times so no cropping allowed).

[feisty2]

then you simply specify a larger "searchparam" if you want a larger radius. also I haven't checked the exact definition of "radius" in mvtools, if it is relative to the block size, then resizing the super clip won't implicitly increase the radius.

[zybex]

Hello to all.

Please tell me the real correct examples of writing the lines necessary for the operation of the commands "MDeGrain6" and "MDeGrainN".

I am using the "MDegrain2" command now in this variant:

super = MSuper (hpad=8, vpad=8, pel=4, levels=0, chroma=true, sharp=2, rfilter=1)
backward_vec1 = MAnalyse(super, blksize=16, search=5, isb=true, delta=1, overlap=8, mt=false)
backward_vec2 = MAnalyse(super, blksize=16, search=5, isb=true, delta=2, overlap=8, mt=false)
forward_vec1 = MAnalyse(super, blksize=16, search=5, isb=false, delta=1, overlap=8, mt=false)
forward_vec2 = MAnalyse(super, blksize=16, search=5, isb=false, delta=2, overlap=8, mt=false)
MDegrain2(super, backward_vec1, forward_vec1, backward_vec2, forward_vec2, thSAD=400, plane=0)

I am satisfied with the numerical values of the parameters prescribed in these lines, I would not want to change them without real need.
But I need to achieve better and / or more accurate noise suppression (perhaps due to the use of additional lines and commands "MDeGrain6" and "MDeGrainN" instead of "MDegrain2").

My source is very noisy.

https://postimg.cc/gallery/MXRr427

[StainlessS]

@Zybex, Any good as example

Posted by Hello_hello

function MDegrainNL(clip c, int "thSAD", int "thSAD2", int "TR", int "BLKSize", int "Overlap", bool "MT", bool "LSB")
{
thSAD = default(thSAD, 150) # Denoising strength
thSAD2 = default(thSAD2, thSAD/2)
TR = default(TR, 1) # Temporal radius
BLKSize = default(BLKSize, 16) # Block size
Overlap = default(Overlap, 4) # Block overlap
MT = default(MT, True) # Internal multithreading
LSB = default(LSB, False) # 16-bit

Super = MSuper(c, MT=MT)
Multi_Vec = MAnalyse(Super, Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, MT=MT)
MDegrainN(c, Super, Multi_Vec, TR, thSAD=thSAD, thSAD2=thSAD2, MT=MT, LSB=LSB)
}

https://forum.doom9.org/showthread.php?p=1861146#post1861146

EDIT: TR seems to govern Temporal Radius, eg 6 for MDegrain6 equiv.

EDIT: Or something like this (untested, everything just repeats)

super = MSuper (hpad=8, vpad=8, pel=4, levels=0, chroma=true, sharp=2, rfilter=1)

backward_vec1 = MAnalyse(super, blksize=16, search=5, isb=true, delta=1, overlap=8, mt=false)
backward_vec2 = MAnalyse(super, blksize=16, search=5, isb=true, delta=2, overlap=8, mt=false)
backward_vec3 = MAnalyse(super, blksize=16, search=5, isb=true, delta=3, overlap=8, mt=false)
backward_vec4 = MAnalyse(super, blksize=16, search=5, isb=true, delta=4, overlap=8, mt=false)
backward_vec5 = MAnalyse(super, blksize=16, search=5, isb=true, delta=5, overlap=8, mt=false)
backward_vec6 = MAnalyse(super, blksize=16, search=5, isb=true, delta=6, overlap=8, mt=false)

forward_vec1 = MAnalyse(super, blksize=16, search=5, isb=false, delta=1, overlap=8, mt=false)
forward_vec2 = MAnalyse(super, blksize=16, search=5, isb=false, delta=2, overlap=8, mt=false)
forward_vec3 = MAnalyse(super, blksize=16, search=5, isb=false, delta=3, overlap=8, mt=false)
forward_vec4 = MAnalyse(super, blksize=16, search=5, isb=false, delta=4, overlap=8, mt=false)
forward_vec5 = MAnalyse(super, blksize=16, search=5, isb=false, delta=5, overlap=8, mt=false)
forward_vec6 = MAnalyse(super, blksize=16, search=5, isb=false, delta=6, overlap=8, mt=false)

MDegrain6(super,
\ backward_vec1, forward_vec1,
\ backward_vec2, forward_vec2,
\ backward_vec3, forward_vec3,
\ backward_vec4, forward_vec4,
\ backward_vec5, forward_vec5,
\ backward_vec6, forward_vec6,
\ thSAD=400, plane=0)

MDegrain6 req 6 pairs of vectors, etc.

[zybex]

@StainlessS

Many thanks.

Your option for "MDegrainN", I'm still in the process of debugging, so far I can’t start working with my parameters there. So far only managed to run with a poor degree of noise reduction.

Your option for "MDegrain6" worked perfectly, the result is much better than with "MDegrain2", noise is significantly better suppressed.


MDegrain2 (plane=0 Luma=On Chroma=Off)
https://i.postimg.cc/ZqJ7DPjw/245-MDegrain2.png (https://postimages.org/)


MDegrain6 (plane=0 Luma=On Chroma=Off)
https://i.postimg.cc/Zqj7k9M4/245-MDegrain6.png (https://postimages.org/)

MDegrain6 (plane=4 Luma=On Chroma=On)
https://i.postimg.cc/gjxjG9gJ/245-MDegrain6-plane-4-Luma-On-Chroma-On.png (https://postimages.org/)

Source
https://i.postimg.cc/2CmMLqt6/245-Source-Color-Gamma-Corrected.png (https://postimages.org/)

[zybex]

I discovered a new condition for the appearance of an unstable result of processing the "MDegrain2" command.

The problem appears in "AVISynth+ x64" (3.6.0 r3276) and "mvtools-2.7.42 x64" (and "2.7.43") when using the "Crop" command in front of a group of commands used by "MDegrain2", and when the external "Multithreading" in "AVISynth+" is turned on, even if the parameter "mt=false" is present in the command "MAnalyse".

Please fix that issue.

I encode in "x264 VFW Lossless" (core 152 r2851bm)
https://www.videohelp.com/download/x264vfw_full_44_2851bm_44825.exe
via "VirtualDub2 x64" build 44282
https://sourceforge.net/projects/vdfiltermod/files/VirtualDub%20pack/version%2020/VirtualDub2_44282.zip/download

The differences are in the heap of different pixels in the heap of different frames.
Visually, these differences are almost imperceptible, but they are, and there are a lot of them.

Example of the difference in frames (comparison through the program "Beyond Compare"):

https://i.postimg.cc/wB7B60k2/2395-Crop-to-416-MDegrain2-Unstable.png (https://postimg.cc/wtYz25ZR)

How to check this problem ?
You make 3 attempts to encode the same AVS file in a row, you get 3 different AVI files. As a rule, their length will be slightly different, already by this sign it can be judged that there are differences in the frames. But even if in two of the three AVI files the length is the same, then anyway, by bitwise comparing them (fc / b file1.avi file2.avi> log), it turns out that they have differences.
If necessary, I can later lay out the instructions on how to find exactly which frames differ in different AVI video files through AVISynth, then you can save them as pictures and compare them visually through "Beyond Compare 3".

==========================================
The original frame size in the file: 720x576 (with black stripes bottom and top).

The processing result is stable, there are no problems - under one of these conditions:
- "Crop" is not used before the group of commands used by "MDegrain2", and "Crop" is not used at all
- "Crop" is used only after a group of commands used by "MDegrain2"
- the entire command block responsible for "MDegrain2" has been deleted, and only the "Crop" command has been left
- disabled "Multithreading" by adding # before "Prefetch (4)"

The processing result is unstable, there is a problem - under one of these conditions:
- Crop is used (to 720x416 or 720x404) in front of a group of commands used by MDegrain2,
with "Multithreading" enabled:
SetFilterMTMode ("DEFAULT_MT_MODE", 2) # at the beginning of the AVS script
Prefetch (4) # at the end of the AVS script

I tried to solve the problem in such ways, but this did not help:
- SetFilterMTMode ("MAnalyse", 3)
- SetFilterMTMode ("MDegrain2", 3)

========================================
Here are the contents of my AVS script:
(I just simplified the script as much as possible for the test so that other commands and filters did not affect the test)

SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\mvtools2.dll")

mpeg2source("The Weekend (feat Shena).d2v")

Crop(0,80,-0,-80) #(left,top,right,bottom)

# this block refers to a filter "MDegrain2"
super = MSuper (hpad=8, vpad=8, pel=4, levels=0, chroma=true, sharp=2, rfilter=1)
backward_vec1 = MAnalyse(super, blksize=16, search=5, isb=true, delta=1, overlap=8, mt=false)
backward_vec2 = MAnalyse(super, blksize=16, search=5, isb=true, delta=2, overlap=8, mt=false)
forward_vec1 = MAnalyse(super, blksize=16, search=5, isb=false, delta=1, overlap=8, mt=false)
forward_vec2 = MAnalyse(super, blksize=16, search=5, isb=false, delta=2, overlap=8, mt=false)
MDegrain2(super, backward_vec1, forward_vec1, backward_vec2, forward_vec2, thSAD=200, plane=0)
# end of block

Prefetch(4)

========================================
Here is my source (DVD VOB music video) - same as before
http://worldclips.ru/clips/Michael+Gray/The+Weekend+%28feat+Shena%29
(you need to log in to download it)

DGDecode.dll - v1.5.8.0 x64
http://www.mediafire.com/?c0wmemj5jam

I use the file "SetFilterMTModes.avsi" in the plugins folder "c:\Program Files (x86)\AviSynth+\plugins64"
from here: https://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest
But even without using it, the problem of instability also exists.

[pinterf]

Thanks, very nice report, reproduced but it's very hard to grab it.
It doesn't even need MDegrain2, reproducible only MShow-ing the first forward vector.
MShow(super, forward_vec1)
It'll take a longer time.

[zybex]

It'll take a longer time.
OK.
Better late than never :-)

[VS_Fan]

DGDecode.dll - v1.5.8.0 x64That's a very old build/version of that plugin. You could use a more recent version: MPEG2DecPlus / D2VSource (http://avisynth.nl/index.php/MPEG2DecPlus). That could help with your problem

[zybex]

That could help with your problem

No, this does not solve the problem.

I changed to "DGSource" from "DGDecodeNV.dll (http://rationalqm.us/dgdecnv/dgdecnv.html)" (build 2053 by 24 May 2020) (instead of "mpeg2source" from "DGDecode.dll"), the problem remains.

[FranceBB]

Hi there.
I was going to perform temporal degrain as usual when I noticed something weird:


source=FFMpegSource2("Cricket.mov", atrack=-1)
soccer=trim(source, 0, 1534)
soccer_fields=AssumeTFF(soccer)
soccer_bob=Bob(soccer_fields)
soccer_cropped=Crop(soccer_bob, 64, 18, -32, -16)
soccer_super=MSuper(soccer_cropped, pel=2, sharp=1)
soccer_bv1=MAnalyse(soccer_super, isb = true, delta = 1, overlap=4)
soccer_fv1=MAnalyse(soccer_super, isb = false, delta = 1, overlap=4)
soccer_bv2=MAnalyse(soccer_super, isb = true, delta = 2, overlap=4)
soccer_fv2=MAnalyse(soccer_super, isb = false, delta = 2, overlap=4)
soccer_degrain=MDegrain2(soccer_super, soccer_bv1, soccer_fv1, soccer_bv2, soccer_fv2,thSADC=200, thSAD=200)
soccer_SD=DebilinearResizeMT(soccer_degrain, 720, 576)


This doesn't work as MSuper doesn't recognize "soccer_cropped" as "clip", therefore MDegrain2 at the end complains about an invalid input.
On the other hand, this works:


source=FFMpegSource2("Cricket.mov", atrack=-1)
soccer=trim(source, 0, 1534)
soccer_fields=AssumeTFF(soccer)
soccer_bob=Bob(soccer_fields)
Crop(soccer_bob, 64, 18, -32, -16)
soccer_super=MSuper(pel=2, sharp=1)
soccer_bv1=MAnalyse(soccer_super, isb = true, delta = 1, overlap=4)
soccer_fv1=MAnalyse(soccer_super, isb = false, delta = 1, overlap=4)
soccer_bv2=MAnalyse(soccer_super, isb = true, delta = 2, overlap=4)
soccer_fv2=MAnalyse(soccer_super, isb = false, delta = 2, overlap=4)
soccer_degrain=MDegrain2(soccer_super, soccer_bv1, soccer_fv1, soccer_bv2, soccer_fv2,thSADC=200, thSAD=200)
soccer_SD=DebilinearResizeMT(soccer_degrain, 720, 576)


Is it intended?
I mean, the first parameter of MSuper is supposed to be clip, right? Why doesn't it accept my custom name?





Side note: to those wondering, I'm not gonna use "Bob", but I use it for quick and dirt previews 'cause otherwise it would take forever for QTGMC to load a preview.

[Boulder]

Does it work with soccer_super=soccer_cropped.MSuper(pel=2, sharp=1) ?

[FranceBB]

Does it work with soccer_super=soccer_cropped.MSuper(pel=2, sharp=1) ?

Yep, it works.

[wonkey_monkey]

There have been some odd things going on with the old "implicit last" in recent version(s) of Avisynth:

https://forum.doom9.org/showthread.php?t=181351&page=32

https://forum.doom9.org/showthread.php?t=181747

[FranceBB]

There have been some odd things going on with the old "implicit last" in recent version(s) of Avisynth:

https://forum.doom9.org/showthread.php?t=181351&page=32

https://forum.doom9.org/showthread.php?t=181747

I see... so it's not filter-specific as it affects other plugins as well... That explains it... Thanks.