MVTools-pfmod

[DTL]

"What does bad blocks mean in that context?"

( blocks[i].GetSAD() > nTh1 )
bad (thSCD1) blocks.

I think it is blocks with SAD > thSCD1 .

[LeXXuz]

I just checked most of my scripts and they are running with mvtools 2.7.44 :eek:

I have no idea why I haven't checked this more thoroughly. :rolleyes:

I already had pinterf's 2.7.45 in my library, no idea why I didn't use it.

DTL you wrote 2.7.45+ so I guess there is already never version to use?

[DTL]

Still no next official release like 2.7.46 from pinterf. With 2.7.45+ I mean post-2.7.45 builds I typically put at https://github.com/DTL2020/mvtools/releases .

I hope someday may be in this or next year :) it will be more or less merged to pinterf branch and after supplementing and testing finally we will see 2.7.46 or may be even 2.8 release. The number of strategic new features and processing modes continue to increase after first fork in 2021. I think to finish adding most of new features after interpolated overlap I plan to start at this week. But after reading mvtools docs one more time at this year :) I found MFlow with per-sample processing precision mode and got new idea to test it with MDegrainN frame-operation pipeline (with easy to use multi-mode). So it may be next more or less slow development feature to the MDegrainN. Also all new features added and approved need to supplement with SIMD processing functions for better speed for most or all supported colour formats and at least some commonly used block sizes and bitdepths. Currently only some formats have at least AVX2 execution in addition to very slow C-reference like only YV12 and only 8x8 block size of 8bit for UseSubShift mode of MVPlane (fully working only in MDegrainN now and planned for MAnalyse and other filters to save RAM and/or better speed at some use cases).

Typically that builds are full featured of this and https://forum.doom9.org/showthread.php?t=183517 thread described features but most of latest builds are with DX12-ME support and will not load where no DX12 installed (so no Win7 and old). So it is required separate build config to build all-win compatible executable without using DX12-ME mode of MAnalyse but using all new features of MDegrainN for example. The overlap single pass MDegrainN for colour formats are also ready to use after 06.07.22 commits but I still not make release build - you can only build yourself from latest git sources image. I plan to make new release after adding the also very strategic interpolated overlap feature to finally get solution against blockiness at some scenes with no-overlap supported hardware accelerated MAnalyse now.

Also the shader need to be fixed after final runtime subshift of block processing implementation based on old mvtools Wiener kernel - it is delayed because 12.05.22 release have not best but about good working SAD compute shader and fast rebuild of shader still not show better results and need more debug.

Some new features like single pass colour MDegrainN or MVLPF interfiltering are not depend on colour format and block size so expected to work already with all old formats. Single pass colour proc also do not require new options so compatible with old scripts directly (expected a bit better speed with same quality).

mvtools will soon be 1/5 of century aged - in January of 2024. So may we will have version 2.8.0 prepared to that celebration. Or may be final release of 2.7.46 :) .

[DTL]

Version for testing of (partial) fix for issue with chroma subsampled formats (both MAnalyse and MDegrainX/N) - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.45-fix1 . The description in https://github.com/pinterf/mvtools/issues/59

One of my test with x264 encoding shows a bit lower output MPEG bitrate and a bit better details at 200% size comparison.

denoise script:
for YV12 source
tr=8
super=MSuper(last,chroma=true, mt=false, pel=2)
multi_vec=MAnalyse (super, multi=true, delta=tr, search=3, searchparam=2, overlap=4, chroma=true, mt=false, truemotion=false, pnew=0, pzero=0, levels=0)
MDegrainN(last,super, multi_vec, tr, thSAD=300, thSAD2=290, mt=false, thSCD1=500)

orig 2.7.45=
x264.exe --profile high --crf 18 --tff --ref 4 -b 10 --me umh --subme 9 --merange 16 --deblock -3:-3 --b-adapt 2 --psy-rd 1.0:0.7 --transfer "bt70
9" --colorprim "bt709" --colormatrix "bt709" -o out.264 src_t.avs
avs [info]: 960x540i 0:0 @ 25/1 fps (cfr)
x264 [warning]: interlace + weightp is not implemented
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64
x264 [info]: profile High, level 3.1, 4:2:0, 8-bit
x264 [info]: frame I:5 Avg QP:14.23 size:102544
x264 [info]: frame P:203 Avg QP:17.24 size: 31718
x264 [info]: frame B:781 Avg QP:20.50 size: 6645
x264 [info]: consecutive B-frames: 1.7% 3.2% 9.1% 11.3% 8.6% 49.1% 4.2% 2.4% 2.7% 3.0% 4.4%
x264 [info]: mb I I16..4: 14.7% 66.3% 19.0%
x264 [info]: mb P I16..4: 1.8% 4.7% 1.0% P16..4: 42.6% 23.4% 19.0% 0.0% 0.0% skip: 7.4%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 31.0% 7.6% 2.1% direct: 7.9% skip:51.3% L0:35.5% L1:32.5% BI:32.0%
x264 [info]: field mbs: intra: 36.6% inter:41.6% skip:31.1%
x264 [info]: 8x8 transform intra:61.4% inter:72.9%
x264 [info]: coded y,uvDC,uvAC intra: 70.9% 82.6% 65.8% inter: 29.9% 25.5% 7.7%
x264 [info]: i16 v,h,dc,p: 29% 19% 15% 36%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 12% 14% 7% 9% 10% 11% 9% 13%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 33% 7% 5% 8% 8% 9% 7% 11%
x264 [info]: i8c dc,h,v,p: 46% 27% 19% 8%
x264 [info]: ref P L0: 49.9% 28.4% 14.0% 4.8% 1.4% 1.4% 0.0% 0.0%
x264 [info]: ref B L0: 70.4% 23.7% 3.4% 1.9% 0.2% 0.2%
x264 [info]: ref B L1: 78.0% 20.4% 0.9% 0.8%
x264 [info]: kb/s:2455.20

encoded 989 frames, 1.42 fps, 2455.20 kb/s

2.7.45 fix1=
x264.exe --profile high --crf 18 --tff --ref 4 -b 10 --me umh --subme 9 --merange 16 --deblock -3:-3 --b-adapt 2 --psy-rd 1.0:0.7 --transfer "bt70
9" --colorprim "bt709" --colormatrix "bt709" -o out.264 src_t.avs
avs [info]: 960x540i 0:0 @ 25/1 fps (cfr)
x264 [warning]: interlace + weightp is not implemented
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64
x264 [info]: profile High, level 3.1, 4:2:0, 8-bit
x264 [info]: frame I:5 Avg QP:14.14 size:102566
x264 [info]: frame P:202 Avg QP:17.24 size: 31792
x264 [info]: frame B:782 Avg QP:20.51 size: 6558
x264 [info]: consecutive B-frames: 1.4% 3.4% 9.7% 8.5% 10.6% 51.6% 4.2% 3.2% 1.8% 1.0% 4.4%
x264 [info]: mb I I16..4: 14.8% 65.9% 19.3%
x264 [info]: mb P I16..4: 1.8% 4.6% 1.1% P16..4: 42.6% 23.5% 19.1% 0.0% 0.0% skip: 7.3%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 30.9% 7.5% 2.1% direct: 7.7% skip:51.6% L0:35.5% L1:32.3% BI:32.1%
x264 [info]: field mbs: intra: 36.0% inter:41.4% skip:31.1%
x264 [info]: 8x8 transform intra:61.4% inter:72.9%
x264 [info]: coded y,uvDC,uvAC intra: 71.4% 82.9% 66.0% inter: 29.7% 25.1% 7.5%
x264 [info]: i16 v,h,dc,p: 30% 20% 15% 35%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 12% 14% 7% 10% 10% 11% 9% 13%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 32% 7% 5% 8% 8% 9% 7% 11%
x264 [info]: i8c dc,h,v,p: 48% 26% 18% 7%
x264 [info]: ref P L0: 49.6% 28.6% 14.2% 4.7% 1.5% 1.4% 0.0% 0.0%
x264 [info]: ref B L0: 70.6% 23.8% 3.3% 1.9% 0.2% 0.2%
x264 [info]: ref B L1: 78.2% 20.3% 0.8% 0.7%
x264 [info]: kb/s:2439.41

encoded 989 frames, 1.42 fps, 2439.41 kb/s


Currently for best results it is recommended to use 2.7.45 and 2.7.45-fix1 in 4:4:4 mode only.

[DTL]

Updated fix02 - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.45-fix02
Found some error in MAnalyse for 4:2:2 and added chroma shift fix to MCompensate to use in QTGMC and other MC-denoise scripts based on MCompensate.

[zorr]

Thanks DTL, your development of new MVTools features and fixes is impressive. Interesting that you found a bug with trymany=true as I've found sometimes trymany=false has the better quality even when it clearly shouldn't.

I haven't tested your version so far but might have some time to do some Zopti investigations. I'm mostly interested in the potential quality improvements of MAnalyse as I have a current use case for it and MCompensate.

Can you provide a list of the new parameters and perhaps a short explanation? That would help constructing appropriate Zopti scripts and make sure I don't miss anything relevant.

[kedautinh12]

I still don't use 2.7.46 ver of DTL cause many change in parameters will affect different result in many scripts base mvtools dependencies

[DTL]

"Interesting that you found a bug with trymany=true"

In my builds of post-2.7.45 in may be my own missing memory init when I add performance optimizations to MAnalyse (skipping already checked predictors positions). Somewhere around end of 2021. I not use trymany=true in that years so not found that bug. In versions 2.7.45 and before all predictors always checked so no need to fill the results vector for each predictor with some 'verybig cost' value because it will be overwritten. After optimization of skipping refining of already checked predictor - some memory were not updated and may keep better cost and result in not correct MV output.

" I'm mostly interested in the potential quality improvements of MAnalyse as I have a current use case for it and MCompensate."

Better check that approach - https://forum.doom9.org/showthread.php?p=1984140#post1984140 multi-generation MVs refining with intermediate denoising by MDegrainN. It gives most of MVs quality boost even at 2nd generation. And it also have main (and secondary) options to zopti-optimizing.

Main params of multi-generation MVs refining are thSAD of first and next MDegrainN. Secondary - may be MAnalyse for 1st and next generations. Currently I think we have too big step in quality after 1st generation and very few in 3 and next. May be better to have more slow quality gain at each generations and it may help to keep more details. So may be the thSAD for 1st MDegrainN may be reduced and the total thSAD(n) vector is subject to check for best quality.
Possible examples of thSAD(generation number) vectors:
1. Current large improvement after 1st generation (400, 200, 200, 200, 200) (result of gen2 is about good so may be used for better performance with stopping iterations after gen2)
2. Possible better quality (250, 240, 230, 220, 200) (result of gen2 lower in quality but gen5 may be better - may be used for better quality with lower performance)

"Can you provide a list of the new parameters and perhaps a short explanation?"

There were update of documentation: https://github.com/DTL2020/mvtools/blob/mvtools-pfmod/Documentation/mvtools2.html . Though still not all new params are listed - mostly about MPB-processing in MDegrainN (still in the very development and some may be removed in future). Also some features support only very few number of block size and bitdepth (other either slow in performance or even crash with error). Mostly checked is my typical used block size 8x8 and 8bit. I will try to add most of new params to March of 2023 to that documentation file soon.

Also there is significant difference already between 'very old 2.7.45' version and current post-2.7.45 builds in features list.

" That would help constructing appropriate Zopti scripts"

That may be complex task even for zopti-robot. Total number of params of MAnalyse+MDegrainN may be >50 now and quality-adjusting on per-scene basis may be >20.
If look into Interface.cpp file https://github.com/DTL2020/mvtools/blob/mvtools-pfmod/Sources/Interface.cpp
MAnalyse args[40]
c[blksize]i[blksizeV]i[levels]i[search]i[searchparam]i[pelsearch]i[isb]b[lambda]i[chroma]b[delta]i[truemotion]b[lsad]i[plevel]i[global]b[pnew]i[pzero]i[pglobal]i[overlap]i[overlapV]i
[outfile]s[dct]i[divide]i[sadx264]i[badSAD]i[badrange]i[isse]b[meander]b[temporal]b[trymany]b[multi]b[mt]b[scaleCSAD]i[optsearchoption]i[optpredictortype]i[scaleCSADfine]f
[accnum]i[UseSubShift]i[SuperCurrent]c[SearchDirMode]i[DMFlags]i

MDegrainN args[43]
ccci[thSAD]i[thSADC]i[plane]i[limit]f[limitC]f[thSCD1]i[thSCD2]i[isse]b[planar]b[lsb]b[thsad2]i[thsadc2]i[mt]b[out16]b[wpow]i[adjSADzeromv]f[adjSADcohmv]f[thCohMV]i
[MVLPFCutoff]f[MVLPFSlope]f[MVLPFGauss]f[thMVLPFCorr]i[adjSADLPFedmv]f[UseSubShift]i[IntOvlp]i[mvmultirs]c[thFWBWmvpos]i
[MPBthSub]i[MPBthAdd]i[MPBNumIt]i[MPB_SPCsub]f[MPB_SPCadd]f[MPB_PartBlend]b[MPBthIVS]i[showIVSmask]b[mvmultivs]c[MPB_DMFlags]i[MPBchroma]i[MPBtgtTR]i[MPB_MVlth]i

+MSuper
c[hpad]i[vpad]i[pel]i[levels]i[chroma]b[sharp]i[rfilter]i[pelclip]c[isse]b[planar]b[mt]b[pelrefine]b

Also MRecalculate may be useful in some scenarios - args[25].
cc[thsad]i[smooth]i[blksize]i[blksizeV]i[search]i[searchparam]i[lambda]i[chroma]b[truemotion]b[pnew]i[overlap]i[overlapV]i[outfile]s[dct]i[divide]i[sadx264]i[isse]b[meander]b[tr]i[mt]b[scaleCSAD]i[optsearchoption]i[optpredictortype]i[DMFlags]i
total is over 100. Thats huge. I understand why people like AI Neural Nework 'auto-magic' tools like RIFE with close to no one adjusting param.

Because MDegrainN is the only and main blending engine in latest 'multi-generation' MVs refining approach - all params of MDegrainN used for the intermediate super clips processing may affect the quality of final output MAnalyse MVs in a sequence.

" I have a current use case for it and MCompensate."

If your consumer filter is MCompensate with 2frames mvclip - you can change last MAnalyse in a multi-generation chain to 2-frames mode (multi=false and set isb and delta params).

Here is drawing how is macro-function of MultiGenerations MG-MAnalyse is designed (for 3 generations of MVs refining):
https://i.ibb.co/MsmVqnc/DSC-1091sm.jpg (https://ibb.co/80H5FMr)
Standard 2.7.45 mvtools MDegrainN output of single pass MAnalyse is 'g1'. Also MAnalyse of 2.7.45 is single input only for single super-clip. With post-2.7.45 builds 2-input MAnalyse used (Ref and Current super-clips inputs).
Each last output of MDegrainN may be used as degrained output or each last output of MAnalyse may be used as mv-clip for mvtools consumer filters.
MV3 out with SuperRef and SrcCurrent may be used with next MDegrainN for g3-degrained output and so on.

I think of creation of schematic drawing of internal MDegrainN design with internal processing and where different params are going. May be sometime later and in some CAD software (it will be large enough drawing).

The 'elementary building block' of in theory infinite generations MVs refinement is 'MDegrain+Super+MAnalyse'. It have inputs of mvclip, super_ref and src. Its output is refined mvclip in current generation.
So it can be designed in function for single step:

Function RefineMV(clip mvclip, clip super_ref, clip src)
{
g_next=MDegrainN(src, super_ref, mvclip, tr, thSAD=my_thSAD, thSAD2=my_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, \
adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel)

return=MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, \
overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, \
global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
}

And usage:
my_src=last
s=Super(my_src)
mv1=MAnalyse(s)
mv2=RefineMV(mv1, s, my_src)
mv3=RefineMV(mv2, s, my_src)
mv4=RefineMV(mv3, s, my_src)
and so on


Also one more param to Zopti check may be tr for each generation. May be SuperCurrent may be named simply super2 because it may be no difference if it current or refs frames to search. Still not tested this idea.

[zorr]

" I'm mostly interested in the potential quality improvements of MAnalyse as I have a current use case for it and MCompensate."

Better check that approach - https://forum.doom9.org/showthread.php?p=1984140#post1984140 multi-generation MVs refining with intermediate denoising by MDegrainN. It gives most of MVs quality boost even at 2nd generation. And it also have main (and secondary) options to zopti-optimizing.

Will do, thanks. Do you think this way is better than the more traditional prefiltering with another denoiser? Have you done comparisons?

Also some features support only very few number of block size and bitdepth (other either slow in performance or even crash with error). Mostly checked is my typical used block size 8x8 and 8bit.

My preliminary tests indicate that block size 32 will work much better than 8, at least in the first phase before MRecalculate. Am I out of luck if I want to use that size? Are the limitations also in the CPU implementation or just the HW accelerated version? Also I recall someone mentioned the overlap of MAnalyse is not supported in all cases?

" That would help constructing appropriate Zopti scripts"

That may be complex task even for zopti-robot. Total number of params of MAnalyse+MDegrainN may be >50 now and quality-adjusting on per-scene basis may be >20.
...
total is over 100. Thats huge. I understand why people like AI Neural Nework 'auto-magic' tools like RIFE with close to no one adjusting param.


Yes, the number of parameters is inhumane. Zopti was created pretty much because I was frustrated with MVTools and wanted an easier way to tune it. So far it has fared well - even with around 50 parameters. It's just a matter of running with a large enough population and iterations. Sometimes I restrict the number of parameters by only optimizing a subset of them and then freezing some parameters and unlocking new ones. Some paramers don't benefit much from trying all possible values, for example you can test LSAD in increments of 100 or even bigger.

" I have a current use case for it and MCompensate."

If your consumer filter is MCompensate with 2frames mvclip - you can change last MAnalyse in a multi-generation chain to 2-frames mode (multi=false and set isb and delta params).

My use case is a bit special - I have two different videos and I want to match the other one to the first one by moulding it as closely as possible to resemble the first one but still maintaining its own characteristics like colours. The second video has pretty much no noise so I think I would only need to use this multi-generation MAnalyse on the first video and get a clean MSuper clip from it before I interleave it with the second video.

I think of creation of schematic drawing of internal MDegrainN design with internal processing and where different params are going. May be sometime later and in some CAD software (it will be large enough drawing).

That would certainly be interesting.

[DTL]

"Do you think this way is better than the more traditional prefiltering with another denoiser? Have you done comparisons?"

The denoiser for each generation may be not only MDegrain if apply previous generation MVs to the next generation denoiser. It is new way of MAnalyse with saving from accumulation of search errors. Still very few comparisons - only with user provided result in https://forum.doom9.org/showthread.php?p=1984142#post1984142 . I typically not use prefiltering because not found it useful. So do not have working scripts for compare.

"My preliminary tests indicate that block size 32 will work much better than 8, at least in the first phase before MRecalculate. Am I out of luck if I want to use that size?"

You may try. At least it is not have SIMD optimized dismetric (SSIM, VIF, covariance) and runtime subshifting functions because too large from SSE2 and AVX2 register file so only slow C-reference. But you can use standard SAD and standard MSuper pelrefining. I not test 32 block size.

"or just the HW accelerated version? "

DX12 search also not support 32 blocksize directly. Yon can only simulate it with feeding /4 divided frame size and setting blocksize of 8 and use MScaleVect after MAnalyse to scale resulted MVs to full frame size. Also may try MRecalculate or onCPU MAnalyse for refine.

"overlap of MAnalyse is not supported in all cases?"

Yes - overlap also not supported with DX12 search modes. There currenly interpolated overlap modes in MDegrain or you can simulate overlap (like diagonal 2x) with scripting - make 1/2 blocksize diagonally shifted copy of clip and use second MAnalyse to create second MVs clip and BlockOverlap pluging to combinr 2 MDegrain clips back.

"for example you can test LSAD in increments of 100 or even bigger."

Yes - some params are float point and need either some not very fine quantization to test not infinite number of points or approaching algorithm to iteratively go to best value like dividing quantization step 2 times around currently found best value.

Full working script example with refineMVs functions now in https://forum.doom9.org/showthread.php?p=1984176#post1984176

[guest]

I still don't use 2.7.46 ver of DTL cause many change in parameters will affect different result in many scripts base mvtools dependencies

Yes, I gave it a try today, and it doesn't work for me :(

[DTL]

Unfortunately it may be not compatible with many old scripts (because of still non-fixed bugs with different block sizes and may be something else). So it is no good to 'install to common plugins folder'. Only load from current working directory for current script.

It was expected pinterf will someday port most important and useful features to more official and final 2.7.46 update. Still waiting. At the begining of 2024 it looks we will have great all planet celebration of 20 years to mvtools so may be some more official release after 2.7.45 expected before January 2024. May be even great version change to 2.8.

Current list of ideas to try still not empty: There is an idea to add more predictors to MAnalyse - to fill big gap between near predictors and global motion predictor. Add 'area' predictors of the bestMV of closest blocks from all previous search levels and median of them. So in the 'all' predictors search mode (optPredictorType=0) area predictors check may be added without refine (and with trymany=true - with all refine) and in reduced predictors mode like (optPredictorType=1) only median predictor (with trymany=true - with refine around this predictor).

Addition: New strategic idea for 202x and later: For trymany=true the AVX2/AVX512 for exhaustive (search=3) may process >1 predictors at single SIMD pass and make significant benefit to current much slower trymany=true mode of MAnalyse. It is close to optSearchOption=3 and 4 idea. But for single current source block and different predictors.

[joka]

I want to re-write some scrips using "multi" Variants. Is there no "multi" MSCDetection? Any alternative / Work arround?

[DTL]

Yes - it looks MSCD can only accept single search pair mvclip. It is really only call to FakePlaneOfBlocks::IsSceneChange() via some steps. multi is a special mode for MAnalyse and I still do not have an ideas if it possible somehow easy to convert multi-mv-clip to pair-mv-clip. Though may be some playing with frame-rate/frame number may help.

In multi mode the MAnalyse select the pair to search and output using special conversion of requested frame number in the required pair:

const int ndiv = (_multi_flag) ? _delta_max * 2 : 1;
const int nsrc = n / ndiv;
const int srd_index = n % ndiv;


If multi enabled - ndiv=_delta_max*2, if disabled (standard pair) - it is 1.

So to request required pair of frame mvs from MAnalyse in multi-mode it required to calculate correct frame number to request and it will output the correct mv frame. But in AVS scripting it may be not very easy (I still not have ideas how to do it - may be play with FrameRate and so on).

Also: MDegrainN still not dead. Completely new processing mode is in development. It is finally motion to the object-oriented moving pictures reconstruction that was expected from AI/neural networks but still no working tools released.

Algorithm (for each block):
1. For the current input pool of blocks in tr-scope - find the most equally (the same) looking block and save it to local memory (with some surrounding area to have ability to perform motion compensation in limited MVs length). Standard thSAD threshold used to separate mostly likely equal blocks from significantly changed. Possible solution - simply save total frame where the 'best' block is found. But it will require very many RAM for storage unknown number of frames (max is total number of blocks in frame to have ability to compensate any MV length, but it looks very rarely required).

2. Analyse current tr-pool of blocks to found best in current pool (+-tr from current frame).

3. Compare best found block in 2. with stored in 1. for visual difference.

3.1. If visual difference (using available dis-metrics - SAD/SSIM/VIF or partial combinations) is below threshold - replace current frame block with stored block (make 100% curing operation).
3.2. If visual difference is above threshold (scenechange, object's view too much change) - replace stored block with current (or best looking in the pool) and output current (or best looked) block.

So for enough denoised clip with first MDegrainN pass it will convert slow motion and slow changed blocks in completely non-changed (with still ability to move around more or less far) and will significantly help to downstream MPEG encoder to mark these blocks as 100% motion compensated and 'skipped'. See skip %-stats in x264 output for example.

So it is threshold-based non-linear processing for complete cut-out of the residual temporal noise over a several frames in a sequence (up to total cut-scene duration).

Something like this already used in modern MPEG encoders (like 265 and may be others) but current implementation is to be used with all possible MPEG encoders like a pre-processing (from MPEG-1 and later). So it is designed to make widely used xvid/x.264 MPEG-4ASP/AVC releases better.

[anton_foy]

Also: MDegrainN still not dead. Completely new processing mode is in development. It is finally motion to the object-oriented moving pictures reconstruction that was expected from AI/neural networks but still no working tools released.
Have you watched Vcmohan's neuralnet and other plugins? He told me that they could help finding temporal differences too. PM him maybe?

Edit: I think now it is called "neural" in "manyPlus" package.

[DTL]

" they could help finding temporal differences "

The task for AI robot denoiser is really very very simple: Using input noised frames sequence _understand_ the scene content (objects, textures, transformations, lighting), create best possible clean textures for all scene objects and simply create scene from scratch using textures and lighting and objects transformations without adding inter-frame noise.

So the main task of denoising is not looking for input noise - it is about looking for clean objects textures (real clean object's views) and simply creating output frames using restored clean textures.

The described above algorithm is some step to 'ideal AI denosier'. It have all elements of this processing: Method to create as clean and as true as possible texture of a block and to create new output frame sequence using only restored textures (if possible without significant distortions).

The main issue of 'linear' MDegrainN processing (it is sort of FIR filter now) - it can not reach non-changing output frame sequence even with large enough and slow in processing tr-value and big thSAD value because at each output frame the total tr-pool of frames advances +1 frame in the input frame sequence so it lost 1 last frame and add 1 next frame and even ideal average blending process will typically create slightly different block result even for ideally motion compensated blocks.
So to create some temporal-thresholding (non-linearity at small enough temporal differences) it is require addition of internal memory for the previous frame output block and some estimatie and resolve algorithm about keep output old block or select new. It looks like IIR filter with possibility to infinite output single and completely same block if some conditions met.

Addition:

First idea about noise measurement: The typical PSNR-type is not work well with very low noise levels still critical to MPEG encoders with high and very high compression ratios (about 1000:1).

So suggested temporal noise measurement in noise-bitrate. In noise bits per second.
Calculation:
Nnps x Ns x FPS
where:
Nnps - number of noise bits per sample
Ns - number of samples in frame (width x height)
FPS - number of frames per second.

So for the 1noise bit per each sample (very close to quantization noise) giving PSNR high above 40 dB the nosie-bitrate for FullHD is still about 50 Mbit/s and MPEG encoder can not go significantly lower in output bitrate without significant distortions. It is also remarkable this noise-bitrate is not depend on bitdepth (so equal in relative amplitude 10bit noise bitrate to 8bit is about 4 times larger). Single frame sequence like Trim(1,1).Loop() have a temporal noise bitrate of true zero.

And 1noise bit per 8bit image already look as 'visually noiseless'. But for MPEG encoder it is still very loaded with bad noise. So much higher denoise ratios required to finally put noise-bitrate good below target MPEG output bitrate (with good enough quality CRF-encoding). So much denoise ratios looks like impossible to reach with 'linear' denoise modes of MDegrainN.

Now made an encoding test: For static camera scene from documentary title with sitting and talking person taking about 10..20% of total frame samples in 1920x1080i 25fps, x264 crf=18 encoding:

After 'visually noiseless' MDegrainN denoising -
2427 Kbit/s,
mb P skip: 40.9%
mb B skip: 76.2%

After creating 1bit BW mask in MS Paint masking talking person with white and all other (about 80% frame) in black and simple scripting:

mask_clip=ImageReader("mask.bmp")
bckgr=Trim(1,1)
bckgr=Loop(bckgr,1000)

Overlay(bckgr, last, mask=mask_clip)


1383 Kbit/s
mb P skip: 85.2%
mp B skip: 93.2%


So with completely static background the number of skipped blocks in MPEG encoder is significantly higher and output bitrate is about 75% lower.

Attempt to apply internal noise-reduction of x264 --nr=600:
1647 K
mb P skip: 37.1%
mb B skip: 77.5%

So even high enough noise-reduction setting for x264 encode can not reach such high skip ratio and so low output bitrate as possible with total block-duplication for non-changed blocks of static parts of the scene.

", the number of parameters is inhumane. "

It is still not very inhumane - it is only some progress from simple denoise application to mv-search engine to more advanced tool. There possible to design control console like that for colour grading pro-work to make realtime adjustments easier (for example some Davinchi products have dedicated hardware control consoles). It is like
https://i.ibb.co/9sNgp7d/mvtools-progr01.jpg (https://ibb.co/V2t9Lsk)

[flossy_cake]

Is it possible to have MVTools output a single floating point value somewhere that defines the overall amount of motion in the scene? I want to use this value in a ScriptClip.

Currently I am using YDiff and/or weighted YUVDiff (like what RT_FrameDifference uses) but I'm finding at times it's not very accurate because of what's in the frame - high APL panning tends to generate higher values, low APL lower values, plus some randomness of the image contents of the frame, eg. if new bright or dark or differently shaped objects are on screen then that could result in a much higher/lower value despite the motion speed being the same.

Thanks

[DTL]

There is some hacky way - convert mvclip to 'frame-clip' (MStoreVect()) and pick dx,dy MVs values for analysis using samples-picking in scripting (also SAD or other dismetric is available). Or some filter redesign required to calculate required value and put to some script-readable form.

It is also way to detect dupe frames - if MStoreVect (cropped-out first line with header data) return 0% zero black frame - it mean all dx=dy=SAD=0 (frames equal).

[flossy_cake]

There is some hacky way - convert mvclip to 'frame-clip' (MStoreVect()) and pick dx,dy MVs values for analysis using samples-picking in scripting (also SAD or other dismetric is available). Or some filter redesign required to calculate required value and put to some script-readable form.

It is also way to detect dupe frames - if MStoreVect (cropped-out first line with header data) return 0% zero black frame - it mean all dx=dy=SAD=0 (frames equal).

Thanks, but I couldn't figure it out. Seems to require 2 passes?

For now I'm using CFrameDiff which is ok - more consistent than YUVDiff in that it still detects motion even when APL is the same. Because of this it's not as good for detecting scenechanges - YUVDiff seems better for that imo. But for motion CFrameDiff is probably the best cheap, fast solution. Some useful thresholds seem to be 0-10=very low motion, 10-25=low motion, 25-45=normal motion, >45=high motion. It also has a fast denoising option in case noise somehow interferes with the metrics.

[DTL]

" require 2 passes?"

No. MAnalyse return MVs data for current stream processed. So you can request any 2 frames from current clip MV analysis at any current frame in clip processing (typically current frame and some other frame, but it may be possible to request any 2 frames from clip - not only current + other).

If you need scenechange detection - it may be not good idea with mvtools. It sort of not search MV data over scenechange at all (it try but result is very bad). The scenechange detection in mvtools is poor enough. It simply look on 'quality of MVs' data as best dismetric over found blocks and if it is very bad over most of frame - it looks like attempt to search for motion over scenechange point in time axis. So may be no great difference in mean MV with static frames of single scene or many random very bad MVs over scenechange. Also mvtools have already implemented very simple filter like MSCDetect that simply output bool flag from internal scenechange detector used by may be all other internal mvtools filters based on SAD/dismetric analysis as described (using 'global' thSCD1 and thSCD2 values).

Also if you like to implement different algorithm for scenechange detection in scripting - you can also grab SAD/dismetric for each block form pseudo-frame clip output by MStoreVect and try to make your own math analysis.

Also for scenechange detection you can try to use any full-ref image quality analysis tools with 2-frames input (like PSNR/SAD/SSIM/VIF/many_other). If the sim/dissim metric over checked pair of frames is very bad - so it looks like frames from different scenes.

If you try to detect scenechange from MV data it may be same task as currently attempt to implement in MPB detector of 'stable MV data' over time axis. But it is much more complex in compare with simple mean value. If MV data is very bad it mean MVs have and random lengths and random directions (angles). So the algorithm for detection if MVs data is very bad and unnatural for typical good intrascene search need to separate many co-directed MVs in areas from mostly random directed MVs in very noisy and low detailed areas or scenechanges. It sort of measuring noise-level in dx/dy data over total set of MVs from single frame pair search. May be even making Fourier domain transform and analyse the shape of Fourier spectrum - for random white nosie it sort of equal-energy over all valid frequencies and for natural motion it may be mostly low-frequencies energy located. But also total length of MVs must be analysed (for single scene and simple dark frame with noise the MVs typically random too). So advanced scenechange detector may be very complex and math and computing loaded and slow.

I also not understand what you need to measure in motion. The mean MV is sort of global MV vector and it close to zero length for different cases (like random noised static scene and also static clean scene). May be you need mean of MVs length in the current pairs of frames analysed ? It may show some measure of motion in a frame, but for static scene with noise it will also correlate with simple noise level if noise is comparable or more of the real scene objects contrast. So for better motion measure the MVs data may be need to be processed with sort of area-based low-pass filtering to found noise-cleaned areas mean MVs at first.

[flossy_cake]

May be you need mean of MVs length in the current pairs of frames analysed ?

Yes, I think so. I am just trying to get a number which describes how much motion there is between 2 frames. Bigger number = more motion, smaller number = less motion. I need the result to be accessible in an Avisynth script variable so I can make conditional decisions on it in the rest of my script - I am not sure if MVTools was designed for this. Looking at the wiki, it seems MVTools stores all the motion vector information in its own bespoke format which can only be used with other modules of the MVTools package. Or perhaps it can be output to a file on disk and then read that information back in from file in realtime.

[DTL]

You can simply make 2 frames equal (Trim(1,1).Loop()) and after MAnalyse put MStoreVect() and look into output clip in 2D viewer with RGB values display for each sample. Try to use some simple sample format like RGB24. After some header in the first line you will see 0,0,0 for each dx,dy,SAD because frames are equal and no motion.

Now take first (top-left) block of size set in MAnalyse and shift to 1 sample left, right, top, bottom and see how RGB data changes. The blocks scanned from left to right and from top to bottom of frame. So you can found where header data ends and simple contigous list of VECTOR data starts and where 1st block data starts. The data is of 32bit integers so takes 4x8bit values. So you collect RGB triplets from frame returned by MStoreVect in script environment and decode it into 2D array of blocks with their VECTOR structure data (dx, dy, SAD). And use dx and dy to calculate each MV length (or squared to be faster and not take square root of slow op) - Length^2 = dx^2 + dy^2. And calculate mean value of MV length over all blocks in frame (width/blocksizeH x height/blocksizeV).

The analysis data header is fixed and the w and h size of the encoded frame is selected by some algorithm. You can skip header fixed and the rest should be 3x32bit VECTOR data for all blocks. If clip sample format is RGB24 these 32bit values are shifted over RGB24 samples and if set to RGB32 - you may be (or not) lucky to get each dx, dy and SAD into RGB32 sample data or each data will have equal start shift by 8bits.

If someone can access sample data from frame (Eval ? or some other function) - he can make simple enough decoder script to get each block VECTOR data from clip produced my MStoreVect(). And make math calculations with dx,dy and SAD data.

The only helper filter for script writers may be created to make 1st line with header data fixed and start 2nd (or other line if blocks number in line is too small) line in RGB32 format from dx data of 1st block in the frame and so on, So the width of a pseudo-clip frame will be number of blocks in line x 3x32bit data (x3 RGB32). So for 1920 width clip and 8x8 block size blocks number per line is 240 and width of MStoreNiceVect will be 240x3=720 and each RGB32 4bytes samples value will keep dx, dy, SAD values for blocks (VECTOR structure for each block). And number or rows (skipping header-row(s)) will be equal to V-number of blocks. It will make script-decoder more simple.

[anton_foy]

request:
A "Threshold" option.
Very often denosing/degraining needs to be stronger in the darker parts (heavier nosie) and less in the brighter ones (lighter noise).
It would be incredibly benificial IMHO to have this option as a limiter of strength when setting a threshold. From default 1.0 to 0.0 or 255-0 or so.
Preferably a soft threshold like a roll-off curve rather than a hard pointy one.

This can already be achived externally but it is very slow and if made internally in mvtools I think it could be much faster in speed.

example:

bright=smdegrain(tr=3,thsad=200)
dark=smdegrain(tr=7,thsad=500)
mask=levels(32,1.0,190,0,255)
mt_merge(dark, bright, mask)

or a simpler limiting example:

original=last
dark=smdegrain(tr=7,thsad=500)
mask=levels(32,1.0,190,0,200) #merge back some denoised together with original by adding 200 on the last param for better visual appearence.
mt_merge(dark, original, mask)

By now I hope I made my idea clear.
A threshold parameter (max 1.0 min 0.0 or 255-0 to adjust how bright it should limit degraining) and a thresholdlimit/thresholdstrength parameter (max 1.0 min 0.0 to adjust strength of degrain in the thresholded areas).
Strength of degraining/denoising can be argued if to blend with the original (simplest but less visually appealing) or to somehow "choke" the strength like rgtools repair or the "limit" parameter in mvtools (although the current parameter limit=1 seems not to lower the degraining very much?).

Pinterf what do you think about this?:)

Thanks!

[DTL]

Short answer (long may be later):

I think you can start from moving 1 level of AVS usage lower at first: Not level of user-provided script function (SMDegrain) but level of manual script writing using plugin functions.

Most quality (per-sample level) can run only at after-MDegrainX processing. So it is not very benefitical in performance to implement user-script in plugin.

Current suggestion:

Put this idea:
bright=smdegrain(tr=3,thsad=200)
dark=smdegrain(tr=7,thsad=500)
mask=levels(32,1.0,190,0,255)
mt_merge(dark, bright, mask)

In manual script (or ask Dogway to add to SMDegrain):

MDegrainN can run at provided tr-value (if not - it can be easily fixed).

So you better in performance is simply run 1 MAnalyse and several MDegrainN and combine in several layers with masking.

Some like:

super=MSuper()
mv=MAnalyse(tr=max(my_tr1, my_tr2),..)

dg1=MDegrainN(super, mv, tr=my_tr1, thSAD=my_thSAD1)
dg2=MDegrainN(super, mv, tr=my_tr2, thSAD=my_thSAD2)

mask=MyCustonMaskFunc(...)
mt_merge(dark, dg1, dg2)



It will be better in performance because of 1 MAnalyse with max_tr only. Typically MAnalyse is most of time-taking process.

[anton_foy]

Short answer (long may be later):

I think you can start from moving 1 level of AVS usage lower at first: Not level of user-provided script function (SMDegrain) but level of manual script writing using plugin functions.

Most quality (per-sample level) can run only at after-MDegrainX processing. So it is not very benefitical in performance to implement user-script in plugin.

Current suggestion:

Put this idea:
bright=smdegrain(tr=3,thsad=200)
dark=smdegrain(tr=7,thsad=500)
mask=levels(32,1.0,190,0,255)
mt_merge(dark, bright, mask)

In manual script (or ask Dogway to add to SMDegrain):

MDegrainN can run at provided tr-value (if not - it can be easily fixed).

So you better in performance is simply run 1 MAnalyse and several MDegrainN and combine in several layers with masking.

Some like:

super=MSuper()
mv=MAnalyse(tr=max(my_tr1, my_tr2),..)

dg1=MDegrainN(super, mv, tr=my_tr1, thSAD=my_thSAD1)
dg2=MDegrainN(super, mv, tr=my_tr2, thSAD=my_thSAD2)

mask=MyCustonMaskFunc(...)
mt_merge(dark, dg1, dg2)



It will be better in performance because of 1 MAnalyse with max_tr only. Typically MAnalyse is most of time-taking process.

Aaah so clever! Of course this is a great idea thank you. So to implement this into mvtools internally would not be faster you think?

Edit:
I did not know this was possible:
mv=MAnalyse(tr=max(my_tr1, my_tr2),..)

Great, I think actually this will help many in retaining more of the original detail where grain/noise is not as heavy in the highlights.

[DTL]

"this was possible:
Quote:
mv=MAnalyse(tr=max(my_tr1, my_tr2),..)"

Well - it is at least must be possible by logic of mvtools plugins but currently may still not work as it should. But it is really very small fix and the most probable pinterf will agree it.

Current status: If tr of MAnalyse != tr of MDegrainN it may cause MVs frame request mismatch because the requesting pseudo-frame number from MDegrainN to MAnalyse is about current_frame x 2 x tr. So the logic in MAnalyse in multi=true decode required current + ref frame pair to search from N frame requested + multi-flag + tr-provided. If tr of MAnalyse and tr of MDegrainN not match - it currently looks cause not correct processing.

But as I see pinterf already add analysis_data structure to MVclip and if it have (easy to add) the MAnalyse tr-value and take it in MDegrainN - the MDegrainN can encode proper N-frame number to request if even tr-value of MAnalyse and MDegrainN not equal.

" to implement this into mvtools internally would not be faster you think?"

It will be somehow faster because of more processing with single memory read-write data. But to implement it - the completely separate processing mode in MDegrainN need to be added. To make dual-tr and dual-thSADs blocks processing and some blending/mixer based on the internal or better external mask clip provided. Also 2.7.45 MDegrainN have only separated processing functions for luma and chroma planes (so need to double number of processing functions). Or attempt to make even more complex one_for_all (templated for better performance ?) processing function of single and dual params processing depending of the current running MDegrainN mode.

It require to add lots of new control params to MDegraiN like:
1. Enable/disable new mode
2. Input mask clip (Y8 ? Y16 ?) param
3. tr2
4. thSAD_second (+Chroma)
5. thSAD2_second (+Chroma)
6.... - really many of possible MDegrainN params need to be doubled (also added in possibly ported features from my post-2.7.45 builds).

So it currently much more easier to implement it as 1 MAnalyse + 2 (or more MDegrainN) and custom mask engine + some blending engine in scripting. It is a completely new mode to test and typically only if most users will like to use it for (months ? years ? decades ?) it possibly may be put to compiled form (also with possible minimul new params or some selected params by users practic accumulated in months/years/decades of scripted form usage).

The only simple fix reqiured to run several MDegrainN with different tr-value using single MAnalyse source. It is really benefitical in performance if (!?!) AVS core will cache already requested MVs frames to 2 or more MDegrainN filters running after single MAnalyse MVs-source. But AVS cache around MAnalyse source may be one more unknown variable - it may be limited to some tr-values only and depend on Prefetch() local and/or global script settings ? I not sure how many MV-frames after MAnalyse will be cached depending on AVS core settings and may be other config params somewhere. But may be for tr=3 all required MV-frames will be cached at most of use cases.

" a simpler limiting example:
Code:
original=last
dark=smdegrain(tr=7,thsad=500)
mask=levels(32,1.0,190,0,200) #merge back some denoised together with original by adding 200 on the last param for better visual appearence.
mt_merge(dark, original, mask)"

This idea may be easier to add as new post-processing engine in addition to 'limit' current post-processing. So it may be programmed as separated module and not touch internal MDegrainN core (of any version). But it may be somehow (may be much) worse in quality in compare with dual-MDegrainN runs with different sets of params. Also may be better to add new input clip param of mask from external mask source (so user will not be limited with internal masking only). But also it may be not significantly faster (additional postprocessing pass really again read-write all frame memory) and only may save some RAM in AVS caching.

[anton_foy]

It require to add lots of new control params to MDegraiN like:
1. Enable/disable new mode
2. Input mask clip (Y8 ? Y16 ?) param
3. tr2
4. thSAD_second (+Chroma)
5. thSAD2_second (+Chroma)
6.... - really many of possible MDegrainN params need to be doubled (also added in possibly ported features from my post-2.7.45 builds).

Sure there could be alot of params depending on this but as I figure only "tr" would suffice. Maybe "thSAD" aswell but I do not think it is needed. But even the limiting by merging the original with mdegrain by a brightness threshold would be great. As you say, the best quality would be two different mdegrain calls merged, yet I have to try the benchmarks to see if it will be alot slower than only one mdegrain call.

Edit: and there is already mmask in mvtools, so why not lumamask :D

[DTL]

"Maybe "thSAD" aswell but I do not think it is needed."

It is typically still work as major protection from too bad blends (causing blurring) where transform compensation is not perfect. So at the lower-nosied medium and high code values it is expected to have both lower tr and lower thSAD - so it will save more details from typicaly possible blurring.

[anton_foy]

"Maybe "thSAD" aswell but I do not think it is needed."

It is typically still work as major protection from too bad blends (causing blurring) where transform compensation is not perfect. So at the lower-nosied medium and high code values it is expected to have both lower tr and lower thSAD - so it will save more details from typicaly possible blurring.

Okay fair enough :)

[DTL]

One more idea with possibly fastest implementation in processing and simple enough in design:

If quality of block-based weighting of current (source, full input noise and details) frame in output blending result will be acceptable - it can be added reading of simple one more input mask clip (in full-frame size or blocks number H and V size only) and simply additionally correct central/current block weight in blending weights after DegrainWeight() call and before normal_weights() call. The correction may be inverse multiplication of all ref-frames blocks between current weight after DegrainWeight() call and zero weights. So in mask clip range 255..0 may be mapped to 1.0..0 float additional multiplier to all ref-frames weights except current frame weight. (May be with additional check if current frame weight !=0). So if mask is 0 - all ref frames weight will be put to zero and norm_weights() will set current frame block only to output (non-changed input). It will make close to zero penalty to performance and keep total implementation very simple.

Also it will make MDegrainN more flexible to use in complex scripts - it someone can make mask of badly blurred areas (blocks) - this mask can be applied as additional input to fix weights of badly transform-compensated ref frames and cure these areas from blurring at next MDegrainN processing (or at final output if mask will be ready before it).

It will be close to

original=last
dark=smdegrain(tr=7,thsad=500)
mask=levels(32,1.0,190,0,200) #merge back some denoised together with original by adding 200 on the last param for better visual appearence.
mt_merge(dark, original, mask)

processing but block-based (not sample-based).

(Registered as feature number 44 in my list of post-2.7.45 build features).

[anton_foy]

One more idea with possibly fastest implementation in processing and simple enough in design:

If quality of block-based weighting of current (source, full input noise and details) frame in output blending result will be acceptable - it can be added reading of simple one more input mask clip (in full-frame size or blocks number H and V size only) and simply additionally correct central/current block weight in blending weights after DegrainWeight() call and before normal_weights() call. The correction may be inverse multiplication of all ref-frames blocks between current weight after DegrainWeight() call and zero weights. So in mask clip range 255..0 may be mapped to 1.0..0 float additional multiplier to all ref-frames weights except current frame weight. (May be with additional check if current frame weight !=0). So if mask is 0 - all ref frames weight will be put to zero and norm_weights() will set current frame block only to output (non-changed input). It will make close to zero penalty to performance and keep total implementation very simple.

Also it will make MDegrainN more flexible to use in complex scripts - it someone can make mask of badly blurred areas (blocks) - this mask can be applied as additional input to fix weights of badly transform-compensated ref frames and cure these areas from blurring at next MDegrainN processing (or at final output if mask will be ready before it).

It will be close to

original=last
dark=smdegrain(tr=7,thsad=500)
mask=levels(32,1.0,190,0,200) #merge back some denoised together with original by adding 200 on the last param for better visual appearence.
mt_merge(dark, original, mask)

processing but block-based (not sample-based).

(Registered as feature number 44 in my list of post-2.7.45 build features).

Great idea. So that could minimize the need for time consuming 2nd+ generation of mdegrain by instead merging it as you explain?

[DTL]

It is single pass MDegrainN as it run today. But it still require some external engine to generate mask clip. Simple mask may be inverted Y channel (and for best performance first downsize to H and V block numbers size). Having block-number sized mask clip also skips possible questions about sample value to use if several samples per block provided. So may be put strict requirement for mask clip to be sized of H and V blocks number. So single Y8 sample per block to use.

In the above examples the 2nd MdegrainN is not 2nd generation - is 2nd parallel running with single MVs source. But that method expected to provide better quality.

[anton_foy]

It is single pass MDegrainN as it run today. But it still require some external engine to generate mask clip. Simple mask may be inverted Y channel (and for best performance first downsize to H and V block numbers size). Having block-number sized mask clip also skips possible questions about sample value to use if several samples per block provided. So may be put strict requirement for mask clip to be sized of H and V blocks number. So single Y8 sample per block to use.

In the above examples the 2nd MdegrainN is not 2nd generation - is 2nd parallel running with single MVs source. But that method expected to provide better quality.

Yes I see but still gains better quality and you agree the blending/merging is a great feature to implement?
Yet it can even be 2 mdegrain calls stacked to get 2nd generation while a brighter is only using 1st gen to use for the brights?

[DTL]

"Yes I see but still gains better quality and you agree the blending/merging is a great feature to implement?"

I agree it can be useful feature to implement.

"Yet it can even be 2 mdegrain calls stacked to get 2nd generation while a brighter is only using 1st gen to use for the brights?"

Yes - we can serialize 2 mdegrains with single MV source and different params and use 1st and last output as inputs to some weighted mixer. There are many ways to implement required processing and it is better to test first in easy to design scripting form the good or better in quality.

[Fjord]

@DTL: Thanks for your development work on MVTools!

Will your multi-generation MV approach work with the existing v2.7.45 MVTools? Or are new processing and/or parameters in your v2.7.46_xxx work-in-progress essential for multi-generation MVs to work?

Your "RefineMV()" function includes new parameters in calls to MDegrainN() and MAnalyse() - such as wpow, adjSADzeromv, adjSADcohmv, ... I can see the variable types in interface.cpp on github, and mention of some of them in your new_features_list.ods spreadsheet, but I haven't yet found any description of these new parameters (purpose, valid values, default value). Do you have some basic description for your new parameters somewhere that I have missed?

I have only tried your latest release v2.7.46_e.02.zip (dated 20230811), with understanding this only contains "compatible" changes to v2.7.45 version. Apparently _e.02 does not include the new parameters used in your multi-generation MV approach, since I get avisynth script errors like "MAnalyse does not have a named argument "optSearchOption".

I have 2k 10-bit YUV422 source videos (high-noise/grain film scans), where BlkSize 32 and 64 are actually useful. But from reading your posts I understand you have only implemented 8-bit operations with 8x8 Blksize. Is that correct? If so I need to stay with MVTools v2.7.45. (and that is why I ask if multi-generation MV approach will work with pinterf's release.) My application uses MCompensate() where refined MVs might help.

[DTL]

"Will your multi-generation MV approach work with the existing v2.7.45 MVTools? "

Possibly not because it is based on MAnalyse with separated inputs for current and ref clips (frames).

In august 2023 I made some special builds tagged as 2.7.46-e.xx - they are based on the latest sources from the pinterf repository (with some commits after 2.7.45 release) and included very few of new and important features. Latest is e.02 - https://github.com/DTL2020/mvtools/releases . So expected to be stable close to 2.7.45 release and have some few possibly not critical to stability features. It has SuperCurrent input for MAnalyse so can be used in multi-generation approach as described.

"Or are new processing and/or parameters in your v2.7.46_xxx work-in-progress essential for multi-generation MVs to work?"

Most important feature is the dual-input MAnalyse filter. Other features of post-2.7.45 builds may add to quality (like MVLPF processing) but not act as the key component of the multi-generation refining idea.

"Your "RefineMV()" function includes new parameters in calls to MDegrainN() and MAnalyse() - such as wpow, adjSADzeromv, adjSADcohmv, ... I can see the variable types in interface.cpp on github, and mention of some of them in your new_features_list.ods spreadsheet, but I haven't yet found any description of these new parameters (purpose, valid values, default value). Do you have some basic description for your new parameters somewhere that I have missed?"

I made some update of HTML documentation but it still not include all latest features added - https://github.com/DTL2020/mvtools/blob/mvtools-pfmod/Documentation/mvtools2.html . Also typically every new added param described in the release description where it was added. So one way to found at least short description is to look into every release description also some extended description sometime added in the posts in the thread with new releases announce - https://forum.doom9.org/showthread.php?t=183517

"I have only tried your latest release v2.7.46_e.02.zip (dated 20230811), with understanding this only contains "compatible" changes to v2.7.45 version. Apparently _e.02 does not include the new parameters used in your multi-generation MV approach, since I get avisynth script errors like "MAnalyse does not have a named argument "optSearchOption"."

You can skip all new params except supercurrent clip for MAnalyse.

"only implemented 8-bit operations with 8x8 Blksize. "

Some of the many new features have only implementation or SIMD faster implementation or were tested with 8x8 block size and 8bit. To display all new features and compatibility with different blocksize/bitdepth combinations someone needs to build a table of current state.
The only required for multi-generation approach feature of dual-input MAnalyse (added SuperCurrent clip input) expected to be compatible with any blocksize/bitdepth.

Expected smallest working RefineMV() function with e.02 build is

Function RefineMV(clip mvclip, clip super_ref, clip src)
{
g_next=MDegrainN(src, super_ref, mvclip, tr, thSAD=my_thSAD, thSAD2=my_thSAD2, mt=false, thSCD1=my_thSCD)

super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel)

return=MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, \
overlap=0, chroma=true, mt=false, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global)
}

And usage:
my_src=last
s=Super(my_src)
mv1=MAnalyse(s)
mv2=RefineMV(mv1, s, my_src)
and so on


"(high-noise/grain film scans), where BlkSize 32 and 64 are actually useful"

I think 'large' blocksizes really works better on film scans because typically they are not only grainy but also very soft. So blocksize 8x8 have too few details to use in good search. And the higher the filmscan resolution the more softer the digital result (and larger blocksize recommended). But for refining of motion after first search stages with large blocksize - the refining MVs with MRecalculate to lower blocksize may be recommended.

[Fjord]

Thanks DTL for the detailed reply. I will look at your revised MVTools2.html and file history for info on new features and parameters.

Thanks for the minimal multi-generation MV script, for your v2.7.46_e.02 (20230811) release. I will give it a try.

[DTL]

"and file history for info on new features and parameters."

It looks I was lazy to put description in single history file - so only available in github releases description (in each version release). https://github.com/DTL2020/mvtools/releases . Will try to collect descriptions from each release in single history file later.

In the script example my_* params are script 'globals' - I found them useful when adjusting MAnalsye for degraining. Also overlap may be set to > 0 for better quality (and even lower performance). I left overlap=0 from my newer versions where I use interpolated overlap in MDegrainN. Really best quality is with full 'real' 4x overlap in MAnalyse (when overlap = bloksize/2).

With high quality settings like trymany=true and overlap=4 (and blksize default of 8x8) even 2 generatins of MVs (single RefineMV call) takes about 10 days of processing 2 hours FullHD film at i5-9600K CPU. And onCPU MAnalyse gives better quality in compare with hardware from GTX1060 card.

Also remember the MAnalyse in refining stages with partially degrained one of 2 input clips gives lower SAD values so the last MDegrainN (or even each next RefineMV call if several present) need to have lower thSAD values for MDegrainN to keep from more blurring. Here the new Auto-thSAD feature for MDegrainN expected to make good help. See example in https://forum.doom9.org/showthread.php?p=1990642#post1990642

[Fjord]

@DTL - I copied all your release descriptions into a Word document for my own reference. I sent you a PM with a link to it.

[DTL]

I added file to github.

Current research question for multi-generation MVs refining (at least using MDegrainN only - not with external denoisers as prefilters):

Is usage of several generations with slow increasing of denoise params produces better MVs in comparison with simple 2 stages with full power denoise params ?

It requires lots of test runs and checks for quality.

Example pseudo-scripts:
1. 2 stages full power denoise in each:

my_src=last
s=Super(my_src)
mv1=MAnalyse(s)
mv2=RefineMV(mv1, s, my_src) // tr=12, thSAD_a=1.3 for MAnalyse/MDegrainN in RefineMV()
final_denoise=MDegrainN(my_src, s, mv2,..) // tr=12, thSAD_a=1.3 for MDegrainN in last output denoise



2. example of several stages with slow incresing of denoise params in refining stages

my_src=last
s=Super(my_src)
mv1=MAnalyse(s)
mv2=RefineMV(mv1, s, my_src) // tr=1, thSAD_a=0.9 for MAnalyse/MDegrainN in RefineMV()
mv3=RefineMV(mv2, s, my_src) // tr=3, thSAD_a=0.9 for MAnalyse/MDegrainN in RefineMV()
mv4=RefineMV(mv3, s, my_src) // tr=6, thSAD_a=1.0 for MAnalyse/MDegrainN in RefineMV()
mv5=RefineMV(mv4, s, my_src) // tr=12, thSAD_a=1.3 for MAnalyse/MDegrainN in RefineMV()
final_denoise=MDegrainN(my_src, s, mv5,..) // tr=12, thSAD_a=1.3 for MDegrainN in last output denoise


better to supplement RefineMV() script with user-provided arguments of tr and thSAD* (_a) params.

Current idea for many generations refining - the first generations of internal degrain with low degrain params (low tr and low thSAD) may lower degrade/blur details so may save more MVs from errors in first stages and provide more correct intermediate denoised clip for next stages of MAnalyse. But this idea check may require lots of test runs.

[tormento]

I added file to github.
Would you please keep on releasing AVX version too?

My PC is quite old and can't deal with AVX2. :p

[DTL]

Here is next release build of possbly stable and compatible -e.xx branch - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-e.03 . Also merged partial fix for unbiased division when work with subsampled colour formats (from Fix02 release of Mar 08 2023). Included SSE2, AVX and AVX2 targeted builds.

[tormento]

Here is next release build
Thank you!

Could you release AVX DirectX12 build too?

[DTL]

I found that branch builds looks like cause process memory corruption and it cause crash in random modules (like ffms2 or AVS core and so on). Will try to found and fix source of the bug.

[tormento]

I found that branch builds looks like cause process memory corruption and it cause crash in random modules (like ffms2 or AVS core and so on). Will try to found and fix source of the bug.
Never had a problem with previous preview version.

How to reproduce?

[DTL]

It looks like we finally have 20-years of MVtools. The first release dated from 0.1-0.6 - 24.01.2004 - 01.04.2004 by Manao. First versions of MotionVectors (Motion.dll).

Still lots of ideas to implement waiting for developers.

[FranceBB]

Oh! It's 20 years already! You're right!
Happy 20th Birthday MVTools!! :D

https://i.imgur.com/0XyxBBJ.png

MVTools has come a long way and it's used pretty much everywhere and it's the core behind so so so many things, be it for linear interpolation or temporal degrain or quite literally any use of Motion Compensation vectors.
A big fat thank you to everyone who worked on this including Manao, Fizick, Tsp, TSchniede, DTL, Pinterf and of course to the wider open source community! :D

[pinterf]

Here is a 2024 version of the good old basic mvtools2 and its DePan companions.

Download mvtools2 2.7.46 (classic), DePan and DePanEstimate
https://github.com/pinterf/mvtools/releases/tag/2.7.46

Package contains builds with several C++ compiler flavours. I did not experienced reproducible speed differences with my usual qtgmc script, but YMMV.

Change log (MvTools)

2.7.46 (20240503) *
Recheck and fix build processes for various compilers
(Visual Studio MSVC v143, v141_xp; Intel C++ Compiler 2024.1 ICX, 19.2 ICL; ClangCL; gcc mingw64)
Fix (#56): bug in Cross search
(unreleased 2.7.46 (20230208) test build in #58)
Fix (#58): MFlowFPS memory leak
Fix (#49): lsad 0 caused division-by-zero crash
Optimization: MDegrain1-6 8 bit: Add avx2 code path (already was in code but disabled, now go live)
Optimization: MDegrainN 10-16 bits: add SSE4.1 code path (was: C only)
Optimization: VerticalBicubic interpolation SSE4.1 version besides SSE2
Fix: MRestoreVect was trying to create unaligned frame (crash)
https://forum.doom9.org/showthread.php?p=1955944#post1955944
(#48) Project files/solution: Intel C++ Compiler 2021 (icx) and 19.2 (icl classic) support on Windows
source: pull avstp 1.04 helper files and reapply earlier patches
Source internals: MDegrain: for block size of 4, read exactly 4 bytes instead of 8. (in extreme cases it would read past a valid memory area)
Source internals: Stop using _mm256_zeroupper in avx2, compilers do that automatically.
Affected LLVM builds, because it unoptimizedly saved and restored all ymm registers (slow!)
https://stackoverflow.com/questions/68736527/do-i-need-to-use-mm256-zeroupper-in-2021

Change log (DePan and DePanEstimate)

Moved to Visual Studio 2022, v141_xp and v143 toolset, Intel Compiler ICX 2024.1 and ICL build support

Depan 2.14

Fix: "DepanScenes" plane parameter for YUY2 clips did not work

DepanEstimate 2.11

Throw an error if memory allocation fails


To preempt any questions: this is like the previous 2.7.45 mvtools2 with some fixes. It does not include any new features from DTL’s fork

[Boulder]

Optimization: MDegrainN 10-16 bits: add SSE4.1 code path (was: C only)


With this optimization, is it now preferred to use MDegrainN instead of MDegrain1 or 2?

[pinterf]

Not really, probably the degraining process itself is not the part of mvtools where the specially written SIMD routines help, especially not for small block sizes. As a power user you'll probably try each DLL which is the best for your processor and use case.
And note that when you pass the same values for thSAD and thSAD2 (and thSADC and thSADC2) then MDegrainN calls the original MDegrain1 or 2 or 3, etc... so there is no MDegrainN call at all.