New DirectX 12 Video APIs

[DTL]

I understand having 50+ different adjustments in mvtools for simple denoise work is not very easy for enduser. This is still in development and new complex processing like multi-pass blending (still about half of all existing ideas implemented - only spatial checks only, not temporal yet) require additional (and not very small) set of params. And it can not be separated to external 'filter' because it is all runtime single pass over frame processing to have best performance.

It were expected if some active users may have time to make tests and report some good set of settings for some use cases. But as time shows there are smaller and smaller number of users of AVS so it may take longer and longer time to wait if someone will put some time to make warp script with 'presets' (like SMDegrain). At 202x years the total world situation at this planet and around my living place changes significantly (with much worse perspective) so I currently put more time to prepare for possibly not very nice future and have lower time to put to development of this completely free project. So it was already ideas to write to pinterf about fixing some state of development and list most useful features being implemented more or less completely (not still left in debug state like optSearchOption=3 and 4 - multi-blocks search for onCPU processing in MAnalyse) in 2021..2022 for transfer to his 'main branch' and finally making 'official' releale of version after 2.7.45.

Now as I see some working MC RIFE version for tr up to 12 I have new ideas of adding some more mode to MDegrainN so it can work as blending engine for external source of motion compensated frames providing protection from too bad blends using same SAD (or any other implemented dissimilarity metric) for blocks method of analisys. While performance at GTX1060 card with tr=12 is about 20x times slower in compare with DX12-ME and current MDegrainN.

[anton_foy]

I understand having 50+ different adjustments in mvtools for simple denoise work is not very easy for enduser. This is still in development and new complex processing like multi-pass blending (still about half of all existing ideas implemented - only spatial checks only, not temporal yet) require additional (and not very small) set of params. And it can not be separated to external 'filter' because it is all runtime single pass over frame processing to have best performance.

It were expected if some active users may have time to make tests and report some good set of settings for some use cases. But as time shows there are smaller and smaller number of users of AVS so it may take longer and longer time to wait if someone will put some time to make warp script with 'presets' (like SMDegrain). At 202x years the total world situation at this planet and around my living place changes significantly (with much worse perspective) so I currently put more time to prepare for possibly not very nice future and have lower time to put to development of this completely free project. So it was already ideas to write to pinterf about fixing some state of development and list most useful features being implemented more or less completely (not still left in debug state like optSearchOption=3 and 4 - multi-blocks search for onCPU processing in MAnalyse) in 2021..2022 for transfer to his 'main branch' and finally making 'official' releale of version after 2.7.45.

Now as I see some working MC RIFE version for tr up to 12 I have new ideas of adding some more mode to MDegrainN so it can work as blending engine for external source of motion compensated frames providing protection from too bad blends using same SAD (or any other implemented dissimilarity metric) for blocks method of analisys. While performance at GTX1060 card with tr=12 is about 20x times slower in compare with DX12-ME and current MDegrainN.

Yes! I have to say it you are really brilliant DTL!!! Plenty of progress in such a small amount of time! Now I have to try your implementation of diagonal blocks for MDegrain. Seriously I think many of your implementations users here on the forum (and elsewhere) have wanted for a long time. Best cheers to you again!!!

EDIT: maybe your "best" and "fastest" settings should be put as "DTL=TRUE/FALSE"?

[magnetite]

No worries DTL. I appreciate what you do. Your work has helped me a lot.

If you need some testing done, let me know. I'm not super experienced with testing, but I can help with some things.

[DTL]

Found some VS attempt to use NVOF for motion compensation - https://bitbucket.org/mystery_keeper/vapoursynth-nvof/src/master/readme.txt . Trying to e-mail Asd-g about porting it to AVS for testing (not sure if solution from https://stackoverflow.com/questions/12686545/how-to-leave-a-message-for-a-github-com-user will work) . Not sure what is the minimum chip from NVIDIA is required to NVOF to work.

Support from NVIDIA looks like very limited https://forums.developer.nvidia.com/t/nvidia-optical-flow-vapoursynth-plugin-motion-compensation/218981

[anton_foy]

Found some VS attempt to use NVOF for motion compensation - https://bitbucket.org/mystery_keeper/vapoursynth-nvof/src/master/readme.txt . Trying to e-mail Asd-g about porting it to AVS for testing (not sure if solution from https://stackoverflow.com/questions/12686545/how-to-leave-a-message-for-a-github-com-user will work) . Not sure what is the minimum chip from NVIDIA is required to NVOF to work.

Support from NVIDIA looks like very limited https://forums.developer.nvidia.com/t/nvidia-optical-flow-vapoursynth-plugin-motion-compensation/218981

Wow amazing DTL! Great to see although pretty rough that support is limited. But it is ported and working for VS now?

[kedautinh12]

I don't think so. I remember Asd-g's gpu don't support CUDA (only for NVIDIA gpu) so he only releases plugin support Vulkan

[DTL]

It is not only NVIDIA but may be new enough and expensive only:
https://docs.nvidia.com/video-technologies/optical-flow-sdk/read-me/index.html
NVIDIA Turing and above GPUs

[Reel.Deel]

Found some VS attempt to use NVOF for motion compensation - https://bitbucket.org/mystery_keeper/vapoursynth-nvof/src/master/readme.txt .

The author of the plugin said it sucks: https://github.com/dubhater/vapoursynth-mvtools/issues/60

[DTL]

I do not see tests with real world shot content from the author. Only tests with black background at NVIDIA site.

The definitely good side of NVIDIA optical flow motion compensation that it is product of a professional fulltime job development team (I hope) and hardware manufacturer. I expect it to have better quality in compare with 'simple' motion search engine from the also NVIDIA chip MPEG encoder. Also it may use much more compute hardware resources for better quality and may have some settings to tweak.

So the plugin is only short interface from optical flow API to AVS and expected to be simple and stable. Also with update in hardware and drivers it is expected to have better in the future. While still working via the same plugin with AVS.
Though I still not have Turing or later hardware to test.

[anton_foy]

I found https://github.com/open-mmlab/mmdetection3d and https://github.com/bharathgs/Awesome-pytorch-list which lists the mm3d optical flow .pth. Maybe something interesting?

Edit: this link to convert to onnx to use for mlrt plugin https://github.com/open-mmlab/mmdetection/blob/master/docs/en/tutorials/pytorch2onnx.md

More lighter (faster?) approach: https://github.com/twhui/LiteFlowNet

[DTL]

There is also newer https://github.com/twhui/LiteFlowNet3 .

As I see the main poor point or all these implementations - usage only 2 input images (I1 I2 or I0 I1). It is very poor approach for noise-deformed real images sets we have as input for temporal denoise process. May be many of motion-estimation / optical-flows algorithms were designed to work on clean images.

With 2 only input images the algorithm can not detect real static (or low speed motion) and low contrast low detailed areas deformed by noise only. And start to produce lots of errors motion vectors (of the very large length sometime). The better approach for motion pictures temporal denoising need to take in analysis as large as possible set of input frames to try to understand the shape and position of objecs in a sequence of a frames. May be some enthusiast can compose and send a message (e-mail ?) to many known motion-estimation or optical flow estimation engines designers with a request to make multi-input frames version of engine that can better work with more or less significantly degraded by natural noise image sequence from single scene objects. At github it may be can be made as opening issue with feature-request in each repository. And the training process of the models must be performed at the datasets with added natural (gauss) noise with target result of clean sources before adding noise.

[anton_foy]

Here he is asking about more input images using optical flow for video. Don't know if it is corresponding to what you look for though. https://discuss.pytorch.org/t/classifying-spatio-temporal-data-videos/68922

Edit: Maybe still need to prefilter before optical flow then?

[DTL]

Edit: Maybe still need to prefilter before optical flow then?

No-no. The motion search must be 'Intelectual' itself. And based only on the true-source non-changed frames (they carry most of useful non-distorted data). Same as started with MPB feature (but it still not go in 'temporal' process - work for single current frame only).

The process is really easy but resources consuming:

Imagine you have some low contrast image in a several printed to paper copies. Each copy have added random gauss-distributed noise with zero mean value across each image point in 'temporal' dimension (in a different copy of printed image).

Now you cut each image to small block (like 8x8 samples with regular grid) with scissors and put each cut pieces in separate box (marked 'pieces of copy 1', pieces of copy2, ..., pieces of copy N). Put same cut picture to the box in random shuffled order like broken mosaic.

Next you call an AI robot with Tera/PetaFLOPS engine with GigaBytes memory to your room and ask to take mostly equal looking pieces from each box and arrange in the same location of arrangement grid.

After solving - ask to calculate Average() of the all pieces in each location of a grid and output resulting denoised image.

For moving scene denoising you print noised sequential frames of a cutscene to different sheets of paper - and cut and ask robot to find equal looking parts and get single denoised frame at the output. To get correct moving picture denoised sequence you keep 'current' frame from messing up into box and put its cut pieces into arrangement grid as a reference and ask robot to find mostly equally looking pieces in all other boxes (before and after frames).

That 'find mostly equally looking to current frame' task is the key of the process. But in real life you do not have the clean not-noised 'current' frame as ideal reference to search. So the better process is iterative in 'time':

Robot create first version of denoised frame using motion compensation to 'current noised frame'. Next create several first-generation denoised frames around current and check for the resulting and used motion vectors used (for real life action the motion vectors are not random like white noise Fourier spectrum in time axis). If MVs are no looks good - the MVs are corrected in some way (like MVLPF in some 'linear processing' way for examlpe). After correction of MVs the new set of frames created:

'frames denoise generation 1'. Next is again for each frame search and apply new MVs using 'frames denoise generation 1' as reference. And use initial source frames as source. Next again analyse if motion looks 'natural' and also all found as similair blocks in a sequence of frames looks mostly equal (so the temporal noise with zero mean is removed).

The total 'esa' exhaustive search/process is really not use search of MVs at all but brute-force try of all possible MVs for current block in a frame (the total count of possible MVs is limited) in degraining (averaging) iterations and analyse target conditions of total cutscene frames pool for best matching.

Target conditions of denoising for frame pool of cutscene (sequential set of frames for single scene or single movie):
1. All MVs look naturally (low freqnency enough main energy components of Fourier spectrum and non-equal energy spectrum like random noise for example).
2. All found objects in a frame pool looks mostly equal (lowest dissimilarity metric like lowest SAD or highest SSIM and so on) in each frame (that mean after backward transforms for each frame).

So it is expected from neural network to perform very multi-pass iterative processing over a supplied input pool of frames (+-tr from current frame for long movie) for creating each denoised or motion compensated frame for output.

The best final MVs for final MC and blending for current frame is a product of very multi-generation search over a large set of frames. Not simple 2 input frames as in typical motion search engines.

Motion search and denoise is iterative multi-generation process: Better denoise provide better MVs and better MVs provide better denoise. But all iterations must use same source set of frames as 'mostly true reference'. To prevent from accumulating of errors in both denoise and MVs in multi-generations.

[DTL]

Make a test of multi-generation MAnalyse+MDegrainN search with version from https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.19 release (it have SuperCurrent optional input for MAnalyse):

Script is

tr=6
super=MSuper(last,chroma=true, mt=false, pel=4)
src=last
mv_1_g0=MAnalyse(super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show1=MShow(super, mv_1_g0).Subtitle("input MAnalyse")

multi_vec=MAnalyse(super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
g1=MDegrainN(last,super, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen1=g1

super_g1=MSuper(gen1,chroma=true, mt=false, pel=4)
multi_vec_g2=MAnalyse (super_g1, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)

mv_1_g1=MAnalyse(super_g1, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show2=MShow(super_g1, mv_1_g1).Subtitle("gen1 MAnalyse")

g2=MDegrainN(src,super, multi_vec_g2, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen2=g2

super_g2=MSuper(gen2,chroma=true, mt=false, pel=4)
multi_vec_g3=MAnalyse(super_g2, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)

mv_1_g2=MAnalyse(super_g2, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show3=MShow(super_g2, mv_1_g2).Subtitle("gen2 MAnalyse")

g3=MDegrainN(src,super, multi_vec_g3, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen3=g3

super_g3=MSuper(gen3,chroma=true, mt=false, pel=4)
multi_vec_g4=MAnalyse(super_g3, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)

mv_1_g3=MAnalyse(super_g3, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show4=MShow(super_g3, mv_1_g3).Subtitle("gen3 MAnalyse")

g4=MDegrainN(src,super, multi_vec_g4, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen4=g4

super_g4=MSuper(gen4,chroma=true, mt=false, pel=4)
multi_vec_g5=MAnalyse(super_g4, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)

mv_1_g4=MAnalyse(super_g4, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show5=MShow(super_g4, mv_1_g4).Subtitle("gen4 MAnalyse")

g5=MDegrainN(src,super, multi_vec_g5, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen5=g5

super_g5=MSuper(gen5,chroma=true, mt=false, pel=4)
#multi_vec_g5=MAnalyse(super_g5, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=2)

mv_1_g5=MAnalyse(super_g5, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show6=MShow(super_g5, mv_1_g5).Subtitle("gen5 MAnalyse")



row1=StackHorizontal(show1, show2)
row2=StackHorizontal(show3, show4)
row3=StackHorizontal(show5, show6)
StackVertical(row1, row2, row3)


https://i.ibb.co/nCxGsdy/6gen-01.png (https://ibb.co/6BjMR9S)

All MDegrainN accepts same input current and super clips and only MAnalyse in each generation uses one source from previous generation MDegrainN and one source from input super clip.

Result shows in several generations of refining of MVs as number of generation increases the number of significantly errorneous MVs at low contrast mostly noised static areas slowly decreases. The mostly visible effect in first 1..2 generations.

[anton_foy]

Make a test of multi-generation MAnalyse+MDegrainN search with version from https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.19 release (it have SuperCurrent optional input for MAnalyse):

Script is

tr=6
super=MSuper(last,chroma=true, mt=false, pel=4)
src=last
mv_1_g0=MAnalyse(super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show1=MShow(super, mv_1_g0).Subtitle("input MAnalyse")

multi_vec=MAnalyse(super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
g1=MDegrainN(last,super, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen1=g1

super_g1=MSuper(gen1,chroma=true, mt=false, pel=4)
multi_vec_g2=MAnalyse (super_g1, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)

mv_1_g1=MAnalyse(super_g1, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show2=MShow(super_g1, mv_1_g1).Subtitle("gen1 MAnalyse")

g2=MDegrainN(src,super, multi_vec_g2, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen2=g2

super_g2=MSuper(gen2,chroma=true, mt=false, pel=4)
multi_vec_g3=MAnalyse(super_g2, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)

mv_1_g2=MAnalyse(super_g2, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show3=MShow(super_g2, mv_1_g2).Subtitle("gen2 MAnalyse")

g3=MDegrainN(src,super, multi_vec_g3, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen3=g3

super_g3=MSuper(gen3,chroma=true, mt=false, pel=4)
multi_vec_g4=MAnalyse(super_g3, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)

mv_1_g3=MAnalyse(super_g3, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show4=MShow(super_g3, mv_1_g3).Subtitle("gen3 MAnalyse")

g4=MDegrainN(src,super, multi_vec_g4, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen4=g4

super_g4=MSuper(gen4,chroma=true, mt=false, pel=4)
multi_vec_g5=MAnalyse(super_g4, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)

mv_1_g4=MAnalyse(super_g4, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show5=MShow(super_g4, mv_1_g4).Subtitle("gen4 MAnalyse")

g5=MDegrainN(src,super, multi_vec_g5, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen5=g5

super_g5=MSuper(gen5,chroma=true, mt=false, pel=4)
#multi_vec_g5=MAnalyse(super_g5, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=2)

mv_1_g5=MAnalyse(super_g5, SuperCurrent=super, delta=1, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)
show6=MShow(super_g5, mv_1_g5).Subtitle("gen5 MAnalyse")



row1=StackHorizontal(show1, show2)
row2=StackHorizontal(show3, show4)
row3=StackHorizontal(show5, show6)
StackVertical(row1, row2, row3)


https://i.ibb.co/nCxGsdy/6gen-01.png (https://ibb.co/6BjMR9S)

All MDegrainN accepts same input current and super clips and only MAnalyse in each generation uses one source from previous generation MDegrainN and one source from input super clip.

Result shows in several generations of refining of MVs as number of generation increases the number of significantly errorneous MVs at low contrast mostly noised static areas slowly decreases. The mostly visible effect in first 1..2 generations.

So this is like prefiltering with MDegrain itself? Refining MVs using denoising/filtering only on MAnalyse.
Edit: Would the "SuperCurrent" be used to compare against the prior superclip (prefiltered with Mdegrain) to get a better estimation? But would not Mrecalculate do similar if using the original superclip
like this:


super = MSuper()
Vec = MAnalyse(super,...)
prefilt = MDegrainN(last,super,...)

superfilt = MSuper(prefilt)
Vec1 = MAnalyse(superfilt,...)
Mvec = MRecalculate(super,Vec1)
MDegrainN(src,super,mvec,...)

[DTL]

"Would the "SuperCurrent" be used to compare against the prior superclip (prefiltered with Mdegrain) to get a better estimation? "

I think no. The total idea of 2-inputs MAnalyse is to make search of 'previous generation degrained' block vs input full-true-non-distorted but +grained block. It should save from quick accumulating errors possible to come from 'prefilters'.

If you like to use MRecalculate - the SuperCurrent input can also be easily added there. But typically MRecalculate is used to refine blocksize or use different search params. For simple multi-generation search the single MAnalyse is enough. Also in each generation the params of search (and intermediate and final MDegrain) may be changed.

Old mvtools allow only search inside single input clip (or you need to try to merge different clips into single input with something like Interleave() and look if it cause correct fetching of 'src' and 'ref' frames of different sources). Search inside single clip cause accumulation of errors after 'prefiltering'.

[anton_foy]

The total idea of 2-inputs MAnalyse is to make search of 'previous generation degrained' block vs input full-true-non-distorted but +grained block. It should save from quick accumulating errors possible to come from 'prefilters'.

Sorry I am a little slow. Just understood it as prefiltering with MDegrain and only using that data to compare against the non-altered data using this line:

multi_vec_g2=MAnalyse (super_g1, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=0, pzero=0, levels=4)

But anyway Great to see the progress even if I did not understand correctly :)

Edit: Yes I thought wrong about MRecalculate, I think I understand now what you mean with your last line of explanation.

[DTL]

Playing with settings of MAnalyse I found disabled by default trymany option. After enabling it it looks like best search mode quickly providing stable enough MVs field in multi-generation search even with thresholding penalties of zero and new predictors are set to zero. Without trymany even in 5 generations there is no any convergence to some stable MVs field observed (may be some quantization noise play role in non-stability).

But it currently work good only with old SAD dismetric and crashes with divide by zero error somwhere with VIF dismetric - need debug and make new version.

Though enabling trymany at MAnalyse will significantly degrades performance because it enable refine search around each predictor (and total number of predictors around 6 or 7). So it looks may be enabled only in highest quality of MVs is required.

Multi-generation search with trymany enabled and default SAD dissimilarity metric for search best matching block:
https://i.ibb.co/5Rbx6fc/fr38-dmflags1-pn0-pz0-gf-tmt.png (https://ibb.co/jrnZT14)

[anton_foy]

Wow great find! Have you considered to try Zopti with your version of mvtools? Can't wait to try the new version of yours.

[DTL]

Some important note for multi-generation MVs refinement: The thSAD for MDegrain need to be significantly reduced after 1st generation of MAnalyse using first generation of MDegrain output. Because SAD of mostly cleaned 'current' block with input noised block become about 2time lower. So thSAD for intermediate generations and last output MDegrain need to be reduced to about 0.5 of initial.

So better multi-generation MVs refinement is some like:

init_thSAD=400

s1=MSuper()
mv1 = MAnalyse(s1)
dg1 = MDegrain(s1, mv1, thSAD=init_thSAD)

1stgen_thSAD = (int)(init_thSAD/1.8) # divisor - subject to Zopti refine ?

s2=MSuper(dg1)
mv2 = MAnalyse(s2, SuperCurrent=s1) # or (s1, SuperCurrent=s2) - may be not visible difference
dg2=MDegrain(s1, mv2, thSAD=1stgen_thSAD)


Also it was found enabling trymany=true in MAnalyse while good refining zero MVs also may add some significantly bad MVs. So it is planned to add flags for predictors used in trymany mode to skip possibly bad predictors and to make performance visibly better.

[anton_foy]

Some important note for multi-generation MVs refinement: The thSAD for MDegrain need to be significantly reduced after 1st generation of MAnalyse using first generation of MDegrain output. Because SAD of mostly cleaned 'current' block with input noised block become about 2time lower. So thSAD for intermediate generations and last output MDegrain need to be reduced to about 0.5 of initial.

So better multi-generation MVs refinement is some like:

init_thSAD=400

s1=MSuper()
mv1 = MAnalyse(s1)
dg1 = MDegrain(s1, mv1, thSAD=init_thSAD)

1stgen_thSAD = (int)(init_thSAD/1.8) # divisor - subject to Zopti refine ?

s2=MSuper(dg1)
mv2 = MAnalyse(s2, SuperCurrent=s1) # or (s1, SuperCurrent=s2) - may be not visible difference
dg2=MDegrain(s1, mv2, thSAD=1stgen_thSAD)


Also it was found enabling trymany=true in MAnalyse while good refining zero MVs also may add some significantly bad MVs. So it is planned to add flags for predictors used in trymany mode to skip possibly bad predictors and to make performance visibly better.

Love those updates! Seems logical with lowering the thSad in the next step. Does this work using DX12 me?

[DTL]

Yes - MAnalyse with optional SuperCurrent can be used with any optSearchOption value (so including hardware search options). I even think of using >1 HWacc in the system for better performance in pipelined way. So first HWacc making initial analysis and second make refining step.

Later we will have many cheap secondhand old HWaccs capable of DX12-ME so it may be tested. Currently you can try accnum different for MAnalyses:

init_thSAD=400

s1=MSuper()
mv1 = MAnalyse(s1, optSearchOption=5, accnum=1) # use first DX12-ME accelerator in system or accnum=0 ? need testing
dg1 = MDegrain(s1, mv1, thSAD=init_thSAD)

1stgen_thSAD = (int)(init_thSAD/1.8) # divisor - subject to Zopti refine ?

s2=MSuper(dg1)
mv2 = MAnalyse(s2, SuperCurrent=s1, optSearchOption=5, accnum=2) # use second DX12-ME accelerator in system or accnum=1 - need testing
dg2=MDegrain(s1, mv2, thSAD=1stgen_thSAD)


Currently using 2 MAnalyse with single HWacc will drop performance about 2 times.

Also may be combination of 1 external PCI-board DX12-ME acc and build-in into CPU may be tested where avaiable.

Also as I read some NVIDIA boards/chips have >1 MPEG encoder ASIC (?) so may expose >1 full-speed DX12-ME interfaces for applications.

At https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new
# OF CHIPS

# OF NVENC /CHIP

Total # of NVENC

So GeForce GTX 965M > 980M / 980MX Maxwell (2nd Gen) may have 2 full-speed DX12-ME interfaces ?
Also GeForce GTX 960 Ti / 970 / 980 , GeForce GTX 980 Ti , GeForce GTX Titan X
GeForce GTX 1070M / 1080M , GeForce GTX 1070 / 1070Ti, GeForce GTX 1080 , GeForce GTX 1080 Ti, GeForce GTX Titan X / Titan Xp


Same is GeForce RTX 4080 Laptop , GeForce RTX 4080 16GB , GeForce RTX 4090 Laptop , GeForce RTX 4090 - but much more expensive.

Also Titan V - 3 NVENC.

Dogway have GTX 1070? May be good to try to ask for testing 1 vs 2 MAnalyse performance (also accepting different accnum values >0 or >1).

Addition: I not sure if several MPEG encoder ASICs located in single physical board will be switched as different Direct3D12 devices with 'accnum' param. May be environment will auto-spread motion estimation tasks if single board have several task dispatch resources avaialble. So 2-NVENC boards may be simply allow to run 2 MAnalyse with about equal speed with default accum=0.

[DTL]

New release: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.20

Fixed possible bug with trymany in MAnalyse.
Added trymany into optPredictorType=1 mode (zero, global and median predictors only).

Added partial fix for 4:2:x formats processing chroma shift issue for MAnalyse, MDegrainN, MCompensate (may also MRecalculate). With the curernt pel-precision from MSuper.

The multi-generations MVs refining looks like also work very visibly against blurring for complex motion like facial animation.

Cleaned from MShow processing script:

my_DMFlags=1
my_thSAD=300
my_thSAD2=250

my_thSAD_mg=150
my_thSAD2_mg=100

my_thSCD=500

my_global=true
my_pzero=10
my_pnew=10
my_pglobal=10

my_pel=2
my_trymany=true

my_oPT=1

tr=6
super=MSuper(last,chroma=true, mt=false, pel=my_pel)

multi_vec=MAnalyse(super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g1=MDegrainN(super, multi_vec, tr, thSAD=my_thSAD, thSAD2=my_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen1=g1

super_g1=MSuper(gen1,chroma=true, mt=false, pel=my_pel)
multi_vec_g2=MAnalyse(super_g1, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)

g2=MDegrainN(super, multi_vec_g2, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen2=g2

super_g2=MSuper(gen2,chroma=true, mt=false, pel=my_pel)
multi_vec_g3=MAnalyse(super_g2, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g3=MDegrainN(super, multi_vec_g3, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen3=g3

super_g3=MSuper(gen3,chroma=true, mt=false, pel=my_pel)
multi_vec_g4=MAnalyse(super_g3, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g4=MDegrainN(super, multi_vec_g4, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen4=g4

super_g4=MSuper(gen4,chroma=true, mt=false, pel=my_pel)
multi_vec_g5=MAnalyse(super_g4, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g5=MDegrainN(super, multi_vec_g5, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen5=g5

super_g5=MSuper(gen5,chroma=true, mt=false, pel=my_pel)
multi_vec_g6=MAnalyse(super_g5, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g6=MDegrainN(super, multi_vec_g6, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

return Interleave(g6.Subtitle("g6"), g2.Subtitle("g2"), g1.Subtitle("g1"))


Generally gen2 is already much sharper at motion in compare with gen1 (standard MDegrainN). Gen6 sometime look a bit better and sometime more blurry. May be some average good number of generations is between 2 and 6 (or some detail-restoration processing may be added to regain details from high-gen if that frame (area of frame) is sharper).

Frames g1, g2 and g6 2x enlarged with BSpline:
https://i.ibb.co/wYfC1Mp/g1.png (https://ibb.co/wYfC1Mp)
https://i.ibb.co/BsSMLfN/g2.jpg (https://ibb.co/BsSMLfN)
https://i.ibb.co/By2ByTj/g6.jpg (https://ibb.co/By2ByTj)

It was non-field separated interlaced so 2 fields present.

May be somehow this many calls to MSuper/MAnalyse/MDegrainN for each generation of MVs refining can be compacted to some AVS function and make script smaller.

imgsli comparisons:
https://imgsli.com/MTYwODgx
https://imgsli.com/MTYwODgw

[DTL]

Real working script with both accelerator and CPU search and refining functions. For 1920x1080i input.


# Input plugins
LoadPlugin("ffms2.dll")
LoadPlugin("mvtools2.dll")

SetFilterMTMode("DEFAULT_MT_MODE", 3)

my_thSAD=260
my_thSAD2=240

my_thSAD_mg=130
my_thSAD2_mg=120

my_thSCD=500

my_pzero=10
my_pnew=10
my_pglobal=10

my_pel=2
my_thCohMV=5 # 5..8 for pel=2, 10..16 for pel=4 ?
my_trymany=true

my_oPT=1
my_overlap=0
my_IntOvlp=3
my_searchparam=2

my_MPBNumIt=2

my_init_tr=12
my_refine_tr=12

Function RefineMV(clip mvclip, clip super_ref, clip src, int _thSAD, int _thSAD2, int in_tr, int refine_tr, int my_thSCD, int my_pel, bool my_trymany, int my_pnew, int my_pzero, int my_pglobal, \
int my_oPT, int my_overlap, int my_searchparam, int my_IntOvlp, int my_thCohMV)
{
g_next=MDegrainN(src, super_ref, mvclip, in_tr, thSAD=_thSAD, thSAD2=_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=my_thCohMV, \
MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp)
super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel)
return MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=refine_tr, search=3, searchparam=my_searchparam, trymany=my_trymany, overlap=my_overlap, chroma=true, mt=false,\
optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=true, optPredictorType=my_oPT)
}

Function RefineMV_HW(clip mvclip, clip super_ref, clip src, int _thSAD, int _thSAD2, int in_tr, int refine_tr, int my_thSCD, int my_pel, int my_thCohMV)
{
g_next=MDegrainN(src, super_ref, mvclip, in_tr, thSAD=_thSAD, thSAD2=_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=my_thCohMV, \
MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3, UseSubShift=1)
super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel, levels=1, pelrefine=false)
return MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=refine_tr, chroma=true, mt=false, optSearchOption=5, levels=1)
}


FFmpegSource2("1920x1080i.mp4")

AddBorders(0,0,0,72)

noproc=last

SeparateFields()

super_hwa=MSuper(last, mt=false, chroma=true, pel=my_pel, hpad=8, vpad=8, levels=1, pelrefine=false)
super_cpu=MSuper(last, mt=false, chroma=true, pel=my_pel, hpad=8, vpad=8, levels=0, pelrefine=true)

multi_vec_hwa=MAnalyse(super_hwa, multi=true, blksize=8, delta=my_init_tr, overlap=0, chroma=true, optSearchOption=5, mt=false, levels=1)
multi_vec_cpu=MAnalyse(super_cpu, multi=true, delta=my_init_tr, search=3, searchparam=my_searchparam, trymany=my_trymany, overlap=my_overlap, chroma=true, mt=false, \
optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=true, optPredictorType=my_oPT)

multi_vec_cpu2=RefineMV(multi_vec_cpu, super_cpu, last, my_thSAD, my_thSAD2, my_init_tr, my_refine_tr, my_thSCD, my_pel, my_trymany, my_pnew, my_pzero, my_pglobal, my_oPT, \
my_overlap, my_searchparam, my_IntOvlp, my_thCohMV)
multi_vec_hybr2=RefineMV(multi_vec_hwa, super_cpu, last, my_thSAD, my_thSAD2, my_init_tr, my_refine_tr, my_thSCD, my_pel, my_trymany, my_pnew, my_pzero, my_pglobal, my_oPT, \
my_overlap, my_searchparam, my_IntOvlp, my_thCohMV)
multi_vec_hwa2=RefineMV_HW(multi_vec_hwa, super_hwa, last, my_thSAD, my_thSAD2, my_init_tr, my_refine_tr, my_thSCD, my_pel, my_thCohMV)

cpu2=MDegrainN(last,super_cpu, multi_vec_cpu2, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp, MPBthSub=5, MPBthAdd=20, MPBNumIt=my_MPBNumIt, \
MPB_SPCsub=0.5, MPB_SPCadd=1.5, MPBthIVS=2200, showIVSmask=false).Weave().Subtitle("cpu2")
hwa2=MDegrainN(last,super_hwa, multi_vec_hwa2, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp, MPBthSub=5, MPBthAdd=20, MPBNumIt=my_MPBNumIt, \
MPB_SPCsub=0.5, MPB_SPCadd=1.5, MPBthIVS=2200, showIVSmask=false).Weave().Subtitle("hwa2")
hybr2=MDegrainN(last,super_hwa, multi_vec_hybr2, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp, MPBthSub=5, MPBthAdd=20, MPBNumIt=my_MPBNumIt, \
MPB_SPCsub=0.5, MPB_SPCadd=1.5, MPBthIVS=2200, showIVSmask=false).Weave().Subtitle("hybr2")

Interleave(cpu2, hybr2, hwa2, noproc.Subtitle("src"))

#last=hybr2

Crop(0,0,1920,1080)

Prefetch(6)


Examples of both accelerator and CPU search and refining and hybrid mode (accelerator first search and CPU refining). Quality onCPU is a bit better. Full CPU search and refine at i5-9600K and 1920x1080i frame run at about 0.28 fps.
Hybrid mode with GTX1060 and i5-9600K CPU run at about 1.24 fps (pel=4) and 1.75fps (pel=2). Quality is close to full CPU search.
Full accelerator search and refine run only a bit faster (about 1.3 fps with pel=4) and quality is a bit lower of hybrid mode at some scenes.

[DTL]

New version: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.21
Added new processing mode to MDegrainN: MEL (Most Equal Looking) search mode for TTH (Temporal Thresholding).
New params to MDegrainN:

pmode=0 (default) - standard blending, pmode=1 - MEL search and TTH only.

TTH_DMFlags - dismetric flags for estimating blocks difference at TTH compare. Flags 0x01 to 0x20 valid (except 0x08).

TTH_thUPD (0 default, additional thresholding disabled, 100% linear mode, must be >0 for pmode=1) - integer threshold for selection: keep output old in pmode=0 or 'best' in pmode=1 block from memory or update block in memory and output new block. Typical working values expected to be significantly below thSAD (like thSAD/3.. thSAD/4 and less). Startng from 0. 0 mean no blocks from memory used (standard MDegrainN mode - FIR filter).

TTH_chroma - use chroma in TTH dismetric analysis (slower, better quality) or not (faster).

Fixed performance issue with double processing of chroma planes in combined YUV processing with no overlap.

Current testscript:

tr=10
super=MSuper(last, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true)
multi_vec=MAnalyse(super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, overlap=0, optSearchOption=1, optPredictorType=0, chroma=false, mt=false)
ref=MDegrainN(last,super, multi_vec, tr, thSAD=185, thSAD2=170, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, IntOvlp=3)

super2=MSuper(ref, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true)
MDegrainN(ref,super2, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, thSCD1=350, pmode=1, TTH_thUPD=100, IntOvlp=3)


TTH_thUPD may be also enabled in 'standard' blending modes (pmode=0 (default)) too (both overlap and no overlap combined YUV processing). It is (much) faster but may provide somehow lower quality. Currently no motion block tracking available so it is mostly effective for completely static blocks only. Some limited motion tracking expected to be in some future versions.

Complexity of analysis in pmode=1 currently is ~tr^2 so it may use separate tr value (and mvclip created with lower tr value). Quality expected to be ~tr value (probability to found most commonly looking block in the total tr-pool). Param thSAD in pmode=1 also controls initial block skipping when accumulating blocks in analysis pool.

TTH_thUPD is the main param to adjust - the higher its value - the more noise blocks are skipped but too high value may cause 'hanging' blocks visible or motion quality degradation. Setting too high thSAD in pmode=1 also may cause more artifacts.

pmode=1 expected to be 'final cleaning' after initial MDegrainN (also must use new super clip with pre-denoised frames) and if highest quality required. For general everyday encodings may be enough to play with TTH_thUPD param in standard pmode=0.

Last MDegrainN with pmode=1 may or may not use refined mv-clip (for best results best refined mvclip is recommended).

pmode=1 not blend at all - so no degrade details quality with any thSAD. It only additional way to select 'best' looking block in current tr-scope and duplicate it in output frames until visual difference with current frame block is below threshold.

TTH_thUPD may be enabled in any MDegraiN in processing script (in MVs refining and final degrain and final cleanup).
TTH_DMFlags may set any avaialable dismetric for visual difference analysis (SAD - faster, SSIM and VIF - slower) at any MDegrainN with enabled TTH_thUPD or pmode=1.

[DTL]

Some morning quicky implementation of this year idea about noise bitrate estimation to check the degrain quality.

Release 30.03.2023 - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.22

Added computing and displaying of residual noise bits count per frame to MCompensate.
Compute sum of log2 of the samples absolute difference between source and motion compensated output frame of MCompensate. For complete static frame sequence RNB=0 bits/frame. For noise bitrate per second - value should be multiplied to frame rate.

New param to MCompensate: showRNB (default = false).

Usage example:
super=MSuper()
mv=MAnalyse(super)
MCompensate(super, mv, showRNB=true)

Currently only for 8bit sources. Need to offset processing function to templated for HBD support. Can process Y only input clip or YUV/RGB (3 planes present). For >1 planes the sum of all planes is displayed.

Computing part:

for (int y = 0; y < nHeight; y++)
{
uint8_t* pDstFrame = pDst[0] + nDstPitches[0] * y;
uint8_t* pSrcFrame = (uint8_t*)pSrc[0] + nSrcPitches[0] * y + nOffset[0];
for (int x = 0; x < nWidth; x++)
{
iSumNzBits += 32 - __lzcnt(SADABS((int)pSrcFrame[x] - (int)pDstFrame[x]));
}
}


Not applicable to float32 samples directly (need convert to some finite precision integer first <32bit).

Usage example to measure denoise process:

SeparateFields()

fields_orig=last

tr=3

super=MSuper(last, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true)
multi_vec=MAnalyse(super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, overlap=0, optSearchOption=1, optPredictorType=0, chroma=false, mt=false, DMFlags=1)
ref=MDegrainN(last,super, multi_vec, tr, thSAD=185, thSAD2=170, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, IntOvlp=3)

super2=MSuper(ref, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true)
MDegrainN(ref,super2, multi_vec, tr, thSAD=350, thSAD2=340, mt=false, thSCD1=350, pmode=1, TTH_thUPD=100, IntOvlp=3)

super_ref=MSuper(ref)
mv_ref=MAnalyse(super_ref)
rnb_den_ref=MCompensate(super_ref, mv_ref, showRNB=true)

super2=MSuper()
mv2=MAnalyse(super2)
rnb_den=MCompensate(super2, mv2, showRNB=true)

super_orig=MSuper(fields_orig)
mv_orig=MAnalyse(super_orig)
rnb_orig=MCompensate(super_orig, mv_orig, showRNB=true)

StackHorizontal(rnb_den, rnb_den_ref, rnb_orig)

Weave()


Output sample frame
https://i.ibb.co/LkGQgRw/rnb-examp01.jpg (https://ibb.co/LkGQgRw)

Yes - the fileds are blended not very nice. It shows how for static areas the second MDegraiN(pmode=1) decreases noise bitcount about 10 times. First stage denoise about 2.9 times decrease noise bitrate. Addition of secondary non-linear IIR-type filter with memory decreases nosie bitrate about 30 times from source.

Completely (100%) temporal denoised frame sequence for zero calibration is
Trim(1,1)
Loop()

[guest]

Some morning quicky implementation of this year about noise bitrate estimation to check the degrain quality.

Release 30.03.2023 - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.22



Just to confirm, and probably a noobie question, but the 2 variants, DX12 & noDX12, are reliant on Direct-X being installed on the system (or not) ??

[DTL]

If you have Win10 or later and compatible hardware you can use DX12 build. It will not load at Win7 or other without DX12 installed. If you not use DX12 search modes in MAnalyse you can safely use noDX12 build.

[mastrboy]

DTL, I can't get your builds to work at all in Windows11, I have tried all 4 different .dll's in the zip file...

AVSmeter just stops at 0 frames forever, until I hit ctrl+c.
AVSMeter64.exe -o d:\test.avs

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r3973, 3.7, x86_64) (3.7.3.0)

Number of frames: 33304
Length (hh:mm:ss.ms): 00:23:09.054
Frame width: 960
Frame height: 720
Framerate: 23.976 (24000/1001)
Colorspace: YV12

Frame (current | last): 0 | 33303

Virtualdub gives me a cryptic memory violation message:
An out-of-bounds memory access (access violation) occurred in module 'VirtualDub64'...
...reading address FFFFFFFFFFFFFFFF.

Avspmod gives a similear error message:
Traceback (most recent call last):
File "_ctypes/callbacks.c", line 315, in 'calling callback function'
File "avsp.pyo", line 5136, in local_wnd_proc
WindowsError: exception: access violation reading 0xFFFFFFFFFFFFFFFF

I have none of these issues with https://github.com/pinterf/mvtools/releases

I also have no idea how to troubleshoot this other than give you some information about my system and hope you have any idea of what is wrong:
AviSynth+ 3.7.3 (r3973, 3.7, x86_64)
Windows 11 22H2 (22621.1413)

Avisynth script I tested with:
FFVideoSource("test.mkv")
SMDegrain(tr=3, thSAD=400, RefineMotion=false, contrasharp=false, plane=4, prefilter=0, chroma=true)

[DTL]

Unfortunately my builds may be not compatible with many old scripts (using no-default block size of 16x16 and may be some more not tested options). So it still pre-release demos of some features and mostly tested at the examples scripts provided here and typically block size of 8x8. I even make somehow changed QTGMC to use with my builds when I tested deinterlacing.

So it is no good to put this .dll in 'common' folder and recommended to load with LoadPlugin() from current working folder. It is expected may be in some years (in beginning of 2024 it is expected great all planet celebration of 20 years for mvtools) we will have some features ported to 'more official' pinterf or may be other programmer capable to test and bugfix most of supported modes of mvtools. But it still not happen. I going to make some e-table (may be google web docs ?) of all new features and ideas accumulated and partially implemented for post-2.7.45 version with current 'status' and other data for analyse and creating list of mostly important features to port/bugfix.

Also I not use SMDegrain script and make my own denosie scripts based on mvtools only. So I not know what cause crash there. May be some day I will have time to attempt to install SMDegrain and check it with debugger where may be crash with that settings and if it possible to more or less fast to fix it.

For the very first possible solution it is recommended to test with block size of 8x8 (internal default for mvtools).

Though if you use SMDegrain as I read it still not support any new features of post-2.7.45 mvtools so it may be safely to use old 'stable' 2.7.45 build from pinterf. May be still many years until we will have some more stable post-2.7.45 build fully compatible with 2.7.45 processing with default new settings and Dogway will make changes to SMDegrain to use new features.

[takla]

@DTL
Have you seen this
https://devblogs.microsoft.com/directx/preview-agility-sdk-1-710-0/
Is it applicable for mvtools?

[DTL]

I have some strategic idea how may be make current post-2.7.45 version more compatible with old scripts and 2.7.45 build - rename all filters with adding _a to the end (like alpha-state). So it may be possible to load both 2.7.45 and post-2.7.45 mvtools in single AVS environment and only use selected filters from post-2.7.45 if required (also it may be (partially) compatible in between - super and mv clips). Now because of same naming it either not loads or may cause undefined usage of different filters from different .dlls. May be in next build.

"Is it applicable for mvtools?"

About new heaps mode - currently the performance is very few limited by textures upload and backward download of MVs and SAD data is very small in size. About sampling - currently some 'simple' sampling mode used in SAD compute shader (sort of sample(x,y) request as CPU from host RAM do (not possible 'complex 3D' sampling when texture mapping to some virtual triangle or other patch performed). So no update of sampling required and can not help in performance.

[DTL]

New release: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.23

Added denoise mask clip input into MDegrainN. Work only on block-based mode. Must be Y8 format with frame size equal to blocks number to process (including any used overlap mode).

New param to MDegrainN:
dnmask - clip. 0 is full standard denoise, 255 is no denoise (so positive Y-channel can be used as mask to degrain only low brightness levels).

Example script (for IntOvlp=3):
dn_mask1=ConvertToY8()

blksize=8
#int_ ovlp=3
dn_mask_x=dn_mask1.width/blksize
overlap_size=blksize/2
dn_mask_y=(dn_mask1.height-overlap_size)/(blksize-overlap_size)
dn_mask1=BilinearResize(dn_mask1, dn_mask_x, dn_mask_y)

dn_mask1=Levels(dn_mask1, 0, 1, 100, 0, 255, coring=false)

dn_masked=MDegrainN(.., IntOvlp=3, dnmask=dn_mask1)

Added update MEL memory with best (lowest sum of DM table row) block and memory for sum of current stored in IIR memory block.

Real fast way to get block numbers is to feed any sized mask clip and read error message if size is not correct - it will show current blocks number for H and V directions for current used overlap mode.

Simple BilinearResize do not make perfect mask for any overlap mode because even blocks rows shifted to the right to overlap size (typical half block size with max overlap). So better to separate to odd/even rows - shift even rows to the right and combine to frame back. But any overlap processing modes looks like hide these errors with not too large block sizes.

[anton_foy]

@DTL

How is it possible to use mdegrain2 with your version, or is it only possible with tr=1 and mdegrain()?

I would like to make my Clay script to work with your version separately to get a speed boost and also a quality boost and yet keep close to the results I get with the current Pinterf version of mvtools.
But I think I have to restructure the script without using mrecalculate and overlap in manalyse.

Edit: maybe you have any further ideas for improvement in both speed and quality when using your version? Need to make it quite simple with for example HQ=true/false or I will have to add many possible parameters.

[DTL]

"is it only possible with tr=1 and mdegrain()?"

MDegrain2 is tr=2. Yes - all new features only included in MDegrainN.

Also as it was found while testing IIR mode with TTempSmooth - any IIR (with previous frames memory) filter can only run in MT_SERIALIZE AVS+ MT mode correctly. So with any IIR-setting enabled (TTH_thUPD > 0) in current MDegraiN release (up to a.23) also require to manually set MT_SERIALIZE for MDegrainN (with SetFilterMTMode(.., force=true)) and to keep multithreading - only use internal AVSTP-based multithreading (mt=true and use updated avstp.dll from pinterf to save from hangs). Thanks to gods pinterf found and fixed that odd issue in avstp and now mvtools can run again with internal multithreading as it really the only possible MT mode with 'temporal' processing like IIR-filtering enabled. MT_SERIALIZE also auto-activated for MAnalyse if 'temporal' predictor is used for the same reason.

In the next versions MDegrainN will auto-register itself with MT_SERIALIZE if any IIR-setting is activated. Maybe also try to set mt=true too ?

"I would like to make my Clay script to work with your version separately to get a speed boost and also a quality boost and yet keep close to the results I get with the current Pinterf version of mvtools.
But I think I have to restructure the script without using mrecalculate and overlap in manalyse."

Best quality of MVs expected only from multi-generation MVs refining - example also in the
https://forum.doom9.org/showthread.php?p=1987964#post1987964

It is more complex in control because it is required to adjust at least 2 different thSAD for first and next generations. With not very small tr-value for first generation it is expected significant part of noise is removed at first generation so the thSAD for second generation may be about 0.6..0.7 of thSAD of first generation. With perfect noise removed it is expected that the last thSAD is first_thSAD/2. But the best strategy of number of generations of refining and decreasing of thSAD (and may be changing tr-value from lower at first generation to higher at second and next generations) at each generation is subject of research (also may be with Zopti). Also with such research the quality metric better be structure-aware (like SSIM or VIF or other).

Also best quality expected from onCPU MAnalyse and full 4x overlap in both MAnalyse and MDegrainN (it is the slowest mode). So currently very many performance/quality modes are possible.

[DTL]

New version - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.24

Added Auto-thSAD for MDegrainN.

New params to MDegrainN:
thSADA_a (float), default = 0. Multiplier proportional to estimated nosie level
thSADA_b (float), default = 0. Offset to calculated Auto-thSAD.

If both thSADA_a and thSADA_b = 0 - Auto-thSAD is disabled.

Used Auto-thSAD is a scaled and offsetted arithmetic mean of blocks SAD values below thSCD1 (noise_estimate). Next is applied adjusting params:
Auto_thSAD = thSADA_a * noise_estimate + thSADA_b

thSAD2, thSADC, thSADC2 calculated proportionally to provided old params values.

For a typical workflow user must provide both non-default thSADA_a and thSADA_b values. If only thSADA_b provided - it will be equal to static thSAD. Expected start values are thSADA_a = 1.0 and thSADA_b = 10.

Setting thSADA_a < 1.0 will make higher denoise on low noise scenes and lower at high nosie scenes.
Setting thSADA_a > 1.0 will make higher denoise on high noise scenes and higher at high noise scenes.

thSADA_b is a simple additive offset (may be negative too).

Initial release of Auto-thSAD feature for testing.

Example:

MDegrainN(last,super, multi_vec, tr, thSAD=135, thSAD2=120, mt=false, wpow=4, thSCD1=450, thSADA_a=1.05, thSADA_b=5, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, IntOvlp=3)


To provide roll-off slope for thSAD if required - user must set thSAD and thSAD2 (also thSADC and thSADC2 for chroma if required). It may be in some abstract units if Auto-thSAD is enabled (only relative ratio is calculated internally). Also users must take care of correct thSCD1 param for medium and high noised scenes. Only blocks with SAD below thSCD1 are used in noise estimation, so too low thSCD1 will cause either too bad estimation or fallback to 'static thSAD' provided as old params. Also may quickly disable any denoising if all frames will be detected as 'scenechange'.

[anton_foy]

New version - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.24

Added Auto-thSAD for MDegrainN.

New params to MDegrainN:
thSADA_a (float), default = 0. Multiplier proportional to estimated nosie level
thSADA_b (float), default = 0. Offset to calculated Auto-thSAD.

If both thSADA_a and thSADA_b = 0 - Auto-thSAD is disabled.

Used Auto-thSAD is a scaled and offsetted arithmetic mean of blocks SAD values below thSCD1 (noise_estimate). Next is applied adjusting params:
Auto_thSAD = thSADA_a * noise_estimate + thSADA_b

thSAD2, thSADC, thSADC2 calculated proportionally to provided old params values.

For a typical workflow user must provide both non-default thSADA_a and thSADA_b values. If only thSADA_b provided - it will be equal to static thSAD. Expected start values are thSADA_a = 1.0 and thSADA_b = 10.

Setting thSADA_a < 1.0 will make higher denoise on low noise scenes and lower at high nosie scenes.
Setting thSADA_a > 1.0 will make higher denoise on high noise scenes and higher at high noise scenes.

thSADA_b is a simple additive offset (may be negative too).

Initial release of Auto-thSAD feature for testing.

Example:

MDegrainN(last,super, multi_vec, tr, thSAD=135, thSAD2=120, mt=false, wpow=4, thSCD1=450, thSADA_a=1.05, thSADA_b=5, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, IntOvlp=3)


To provide roll-off slope for thSAD if required - user must set thSAD and thSAD2 (also thSADC and thSADC2 for chroma if required). It may be in some abstract units if Auto-thSAD is enabled (only relative ratio is calculated internally). Also users must take care of correct thSCD1 param for medium and high noised scenes. Only blocks with SAD below thSCD1 are used in noise estimation, so too low thSCD1 will cause either too bad estimation or fallback to 'static thSAD' provided as old params. Also may quickly disable any denoising if all frames will be detected as 'scenechange'.

So cool! Does this new feature slow things down alot?

Edit: btw. I tried to make your version correspond visually to pinterf's latest version of mvtools2 but yours with optSearchOption=5 and intOvlp=3 gave less denoising and less temporal stability even if I turned up thsad. Will post the full script comparisons later today if I can (Clay with fast=true since your version does not have MDegrain2 now) . Also I did not get any speed improvement which I guess is because of my old GPU.

[DTL]

"Does this new feature slow things down alot?"

I did not take tests of performance. But it is expected to be very fast and not make a visible performance hit. If performance hit is visible - performance may be better in next versions with pre-calculating of tr-weights. Currently each frame tr-weights roll-off (defined by thSAD/thSAD2 difference) calculated using float cos() function.

" optSearchOption=5 and intOvlp=3 gave less denoising and less temporal stability"

In my tests the quality of ME with the GTX1060 card is somehow worse in comparison with onCPU MAnalyse. But acceptable for the denoise of documentaries series with offloading part of work from CPU so total mvtools+x264 encoding run faster. For highest quality denoise work only onCPU MAnalyse is recommended (optSearchOption != 5/6).

Hardware ME from MPEG encoder ASIC is not simply hardware-accelerated MAnalyse but completely different ME engine may be optimized for faster MPEG encoding and not for quality. Also at each version of hardware and each vendor (NVIDIA/AMD/Intel/others ?) it may provide different quality and performance.

Maybe hardware ME can be used to make things faster in multi-generations ME refining as first generation of MAnalyse.

My current test script for 2 generations MVs refining and Auto-thSAD used:

# Input plugins
LoadPlugin("ffms2.dll")
LoadPlugin("mvtools2.dll")

SetFilterMTMode("DEFAULT_MT_MODE", 3)

my_thSADA_a=1.1
my_thSADA_b=50

my_thSAD=250
my_thSAD2=Int(Float(my_thSAD) * 0.8)

my_thSAD_mg=150
my_thSAD2_mg=Int(Float(my_thSAD_mg) * 0.8)

my_thSCD=my_thSAD+200

my_pzero=10
my_pnew=10
my_pglobal=10

my_pel=2
my_thCohMV=5 # 5..8 for pel=2, 10..16 for pel=4 ?
my_trymany=false

my_oPT=1
my_overlap=0
my_IntOvlp=3
my_searchparam=2

my_MPBNumIt=2

my_init_tr=6
my_refine_tr=6

Function RefineMV(clip mvclip, clip super_ref, clip src, int _thSAD, int _thSAD2, int in_tr, int refine_tr, int my_thSCD, int my_pel, bool my_trymany, int my_pnew, int my_pzero, int my_pglobal, \
int my_oPT, int my_overlap, int my_searchparam, int my_IntOvlp, int my_thCohMV)
{
g_next=MDegrainN(src, super_ref, mvclip, in_tr, thSAD=_thSAD, thSAD2=_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.6, adjSADcohmv=0.6, thCohMV=my_thCohMV, \
MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp)
super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel)
return MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=refine_tr, search=3, searchparam=my_searchparam, trymany=my_trymany, overlap=my_overlap, chroma=true, mt=false,\
optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=true, optPredictorType=my_oPT)
}

Function RefineMVa(clip mvclip, clip super_ref, clip src, int _thSAD, int _thSAD2, float _thSADA_a, float _thSADA_b, int in_tr, int refine_tr, int my_thSCD, int my_pel, bool my_trymany, int my_pnew, int my_pzero, int my_pglobal, \
int my_oPT, int my_overlap, int my_searchparam, int my_IntOvlp, int my_thCohMV)
{
g_next=MDegrainN(src, super_ref, mvclip, in_tr, thSAD=_thSAD, thSAD2=_thSAD2, thSADA_a=_thSADA_a, thSADA_b=_thSADA_b, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.6, adjSADcohmv=0.6, thCohMV=my_thCohMV, \
MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp)
super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel)
return MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=refine_tr, search=3, searchparam=my_searchparam, trymany=my_trymany, overlap=my_overlap, chroma=true, mt=false,\
optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=true, optPredictorType=my_oPT)
}

FFmpegSource2("src.mp4")

noproc=last

super_cpu=MSuper(last, mt=false, chroma=true, pel=my_pel, hpad=8, vpad=8, levels=0, pelrefine=true)

multi_vec_cpu=MAnalyse(super_cpu, multi=true, delta=my_init_tr, search=3, searchparam=my_searchparam, trymany=my_trymany, overlap=my_overlap, chroma=true, mt=false, \
optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=true, optPredictorType=my_oPT)

multi_vec_cpu2=RefineMV(multi_vec_cpu, super_cpu, last, my_thSAD, my_thSAD2, my_init_tr, my_refine_tr, my_thSCD, my_pel, my_trymany, my_pnew, my_pzero, my_pglobal, my_oPT, \
my_overlap, my_searchparam, my_IntOvlp, my_thCohMV)

multi_vec_cpu2a=RefineMVa(multi_vec_cpu, super_cpu, last, my_thSAD, my_thSAD2, my_thSADA_a, my_thSADA_b, my_init_tr, my_refine_tr, my_thSCD, my_pel, my_trymany, my_pnew, my_pzero, my_pglobal, my_oPT, \
my_overlap, my_searchparam, my_IntOvlp, my_thCohMV)

cpu2=MDegrainN(last,super_cpu, multi_vec_cpu2, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp, MPBthSub=5, MPBthAdd=20, MPBNumIt=my_MPBNumIt, \
MPB_SPCsub=0.5, MPB_SPCadd=1.5, MPBthIVS=2200, showIVSmask=false)

cpu2a=MDegrainN(last,super_cpu, multi_vec_cpu2a, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, thSADA_a=my_thSADA_a, thSADA_b=my_thSADA_b, mt=false, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp, MPBthSub=5, MPBthAdd=20, MPBNumIt=my_MPBNumIt, \
MPB_SPCsub=0.5, MPB_SPCadd=1.5, MPBthIVS=2200, showIVSmask=false)

cpu2a_s=MDegrainN(last,super_cpu, multi_vec_cpu2a, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, thSADA_a=my_thSADA_a, thSADA_b=my_thSADA_b, mt=false, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp)

cpu_s=MDegrainN(last,super_cpu, multi_vec_cpu, my_init_tr, thSAD=my_thSAD, thSAD2=my_thSAD2, mt=false, thSCD1=my_thSCD, IntOvlp=my_IntOvlp)

Interleave(noproc.Subtitle("src"),cpu2.Subtitle("cpu2"), cpu2a_s.Subtitle("cpu2a_s"), cpu_s.Subtitle("cpu_s"))

Prefetch(..)


Where interleaved output frames
src - input source
cpu2 - 2 generations MVs refined with MPB and static thSAD
cpu2a_s - 2 generations MVs refined without MPB and Auto-thSAD at all generations
cpu_s - single generation MAnalyse and MDegrain with static thSAD (mostly close to 2.7.45 version, only interpolated overlap used for better performance).

Settings for MAnalsye in the script are not the best possible for best quality - I set lower for better performance at my old test CPU of E7500. Better quality expected with
my_pel=4
my_thCohMV=12 # 10..16 for pel=4 ?
my_trymany=true

my_oPT=0 # all predictors used
my_overlap=4 # full 4x real search overlap - slowest
my_IntOvlp=0
my_searchparam=2 # better expected with >2 and also pelsearch > 4 (for pel=4)

MPB processing in MDegrainN still looks not make things visibly better (at least at my grainy test footage) so currently may be disabled for a bit better performance. 2 generations MVs refining sometime reduce search errors also at the borders of objects and dark parts of scenes. Using of Auto-thSAD (with old added SAD-related tweaks for static and moving and 'coherent moving' blocks with adjSADzeromv, adjSADcohmv keeps more details at some areas like moving parts with lower denoising at these areas).

[tormento]

I hope to see all those news in SMDegrain soon ;)

[DTL]

It is not likely to be soon until we get good programmers to fix currently already added bugs. As I found with an attempt to enable internal MT with avstp.dll - both MAnalyse and MDegrain crashes with something like memory corruption. Only works stable with AVS+ global frame-based MT. So it looks like compatibility with internal MT via AVSTP is severely broken. An internal MT in MDegrainN is highly recommended if use IIR-based temporal additional filtering (only works good in MT_SERIALIZED). So I think Dogway does not like to use such not stable versions.

About using hardware ME with very slow filtering - it really greatly helps in 2 generations MVs refining. 2 MAnalyse with 'very' slow settings like pel=4, tr=12, trymany=true close to no-start at all. And with the use of DX12-ME from GTX1060 at first MAnalyse total transcoding runs at about 0.25 fps with i5-9600K CPU.

Current practical script with 'best quality' settings is:

# Input plugins
LoadPlugin("ffms2.dll")
LoadPlugin("mvtools2.dll")

SetMemoryMax(10000)

my_thSADA_a=1.3
my_thSADA_b=80

my_thSAD=250
my_thSAD2=Int(Float(my_thSAD) * 0.8)

my_thSAD_mg=150
my_thSAD2_mg=Int(Float(my_thSAD_mg) * 0.8)

my_thSCD=my_thSAD+200

my_pzero=10
my_pnew=10
my_pglobal=10

my_pel=4
my_thCohMV=4 # 5..8 for pel=2, 10..16 for pel=4 ?
my_trymany=true

my_oPT=0
my_overlap=4
my_IntOvlp=0
my_searchparam=4
my_pelsearchparam=4

my_MPBNumIt=2

my_init_tr=12
my_refine_tr=12

my_MT=false

Function RefineMVa(clip mvclip, clip super_hwa, clip super_ref, clip src, int _thSAD, int _thSAD2, float _thSADA_a, float _thSADA_b, int in_tr, int refine_tr, int my_thSCD, int my_pel, bool my_trymany, int my_pnew, int my_pzero, int my_pglobal, \
int my_oPT, int my_overlap, int my_searchparam, int _pelsearchparam, int my_IntOvlp, int my_thCohMV, bool _my_MT)
{
g_next=MDegrainN(src, super_hwa, mvclip, in_tr, thSAD=_thSAD, thSAD2=_thSAD2, thSADA_a=_thSADA_a, thSADA_b=_thSADA_b, mt=_my_MT, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.6, adjSADcohmv=0.6, thCohMV=my_thCohMV, \
MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)
super_g_next=MSuper(g_next,chroma=true, mt=_my_MT, pel=my_pel)
return MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=refine_tr, search=3, searchparam=my_searchparam, pelsearch=_pelsearchparam, trymany=my_trymany, overlap=my_overlap, chroma=true, mt=false,\
optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=true, optPredictorType=my_oPT)
}

FFmpegSource2("src.mp4")

super_cpu=MSuper(last, mt=my_MT, chroma=true, pel=my_pel, hpad=8, vpad=8, levels=0, pelrefine=true)
super_hwa=MSuper(last, mt=my_MT, chroma=true, pel=4, hpad=8, vpad=8, levels=1, pelrefine=false)

multi_vec_cpu=MAnalyse(super_cpu, multi=true, delta=my_init_tr, search=3, searchparam=my_searchparam, trymany=my_trymany, overlap=my_overlap, chroma=true, mt=false, \
optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=true, optPredictorType=my_oPT)

multi_vec_hwa=MAnalyse(super_hwa, multi=true, delta=my_init_tr, chroma=true, mt=false, \
optSearchOption=5, levels=1)

multi_vec_cpu2a=RefineMVa(multi_vec_hwa, super_hwa, super_cpu, last, my_thSAD, my_thSAD2, my_thSADA_a, my_thSADA_b, my_init_tr, my_refine_tr, my_thSCD, my_pel, my_trymany, my_pnew, my_pzero, my_pglobal, my_oPT, \
my_overlap, my_searchparam, my_pelsearchparam, my_IntOvlp, my_thCohMV, my_MT)

MDegrainN(last,super_cpu, multi_vec_cpu2a, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, thSADA_a=my_thSADA_a, thSADA_b=my_thSADA_b, mt=my_MT, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp)

Prefetch(..)


But as I see at total transcoding fps about 0.25 and FullHD frame it uses only about 1..2% of hardware encoder. So most of the CPU time looks like sitting in the second MAnalyse with slow settings for best quality.

So I get new ideas about better quality of MVs using still free resources of hardware accelerator: To use extra free resources of hardware ME accelerator (also typically not capable to make overlapping processing in mvtools-order with single search job) send several small steps shifted frames for search MVs with a bit different blocks assignment (like +-1 sample for 4:4:4 formats and +-2 samples for 4:2:0 formats) to generate 4 or 8 additional MVs around 'current' block position and calculate some averaging of these 5 or 9 MVs to get possibly more noise-free MV for current block. Averaging modes may be arithmetic mean or median (or other non-linear filtering of data 1D vector or even 2D array). To make it usable with any MAnalyse mode and any other filter consumer of MVs data - make it finally separated mvtools filter like MVProc() with 5 or 9 possible inputs from several MAnalyse (or in the future 1 input from single MAnalyse in special multi-mode). Also maybe transfer MVLPF (and other possible future MVs data intermediate processing) in this filter so it can be used with any MVs data consumer filter in complex scripting and allow to split its output to different filters using AVS scripting - for example as additional predictor for multi-generation search scripts (see feature 48 also). The number of search positions around the current block may be increased up to filling all possible integer blocks positions. Also maybe subsample shifted positions can be added too (to fill radius of 0.5..0.25 to 1.25 and more around current block position).

Expected new features script is like:

#current block pos
super=MSuper(last)
current_mvclip=MAnalyse(super,..)

#shifted 2 samples up block pos
sh2_up=Crop(0,2,last.width, last.height-2).AddBorders(0,0,2,0)
super_sh2up=MSuper(sh2_up)
sh2_up_mvclip=MAnalyse(super_sh2up,..)

# same here for shifting 2 samples left, down, right

# combine 5 MV clips from current and shifted blocks assignment
mvclip=MVProc(current_mvclip, sh2_up_mvclip, sh2_down_mvclip, sh2_left_mvclip, sh2_right_mvclip, average_mode='median',.., optional MVLPF and other)

MDegrainN(last, super, mvclip,...)


So it still requires some development time to check this idea of MVs refining at the typical complex places like low contrast and heavily noisy areas. Where noise close or above amplitude of texture details so simple MVs search typically greatly fails and it causes details blurring with MDegrain.

[kedautinh12]

Pinterf was fixed bug from AVSTP
https://github.com/pinterf/AVSTP/releases

[DTL]

Yes - so it is usable with AVS+ again. But now it only can run with a 2.7.45 build because I did not test it during the years of development of my version and it looks like several crash-bugs with internal MT using AVSTP was accumulated.

Some test script with 5x overlap with hardware ME (and vsttempsmooth pmode=1 as median-like 'best' sample value select engine at the end):

# Input plugins
LoadPlugin("ffms2.dll")
LoadPlugin("mvtools2.dll")
LoadPlugin("vsTTempSmooth.dll")

SetMemoryMax(10000)

my_thSADA_a=1.1
my_thSADA_b=60

my_thSAD=250
my_thSAD2=Int(Float(my_thSAD) * 0.8)

my_thSAD_mg=150
my_thSAD2_mg=Int(Float(my_thSAD_mg) * 0.8)

my_thSCD=my_thSAD+200

my_thCohMV=4

my_refine_tr=12

my_MT=false

FFmpegSource2("src.mp4")

AddBorders(16,16,16,16)

super_hwa_center=MSuper(last, mt=my_MT, chroma=true, pel=4, hpad=8, vpad=8, levels=1, pelrefine=false)

src=last
shift_val=4

src_up=Crop(0,shift_val,src.width-0,src.height-shift_val).AddBorders(0,0,0,shift_val)
src_down=Crop(0,0,src.width-0,src.height-shift_val).AddBorders(0,shift_val,0,0)
src_left=Crop(shift_val,0,src.width-shift_val,src.height-0).AddBorders(0,0,shift_val,0)
src_right=Crop(0,0,src.width-shift_val,src.height-0).AddBorders(shift_val,0,0,0)

super_hwa_up=MSuper(src_up, mt=my_MT, chroma=true, pel=4, hpad=8, vpad=8, levels=1, pelrefine=false)
super_hwa_down=MSuper(src_down, mt=my_MT, chroma=true, pel=4, hpad=8, vpad=8, levels=1, pelrefine=false)
super_hwa_left=MSuper(src_left, mt=my_MT, chroma=true, pel=4, hpad=8, vpad=8, levels=1, pelrefine=false)
super_hwa_right=MSuper(src_right, mt=my_MT, chroma=true, pel=4, hpad=8, vpad=8, levels=1, pelrefine=false)

mv_hwa_center=MAnalyse(super_hwa_center, multi=true, delta=my_init_tr, chroma=true, mt=false, optSearchOption=5, levels=1)
mv_hwa_up=MAnalyse(super_hwa_up, multi=true, delta=my_init_tr, chroma=true, mt=false, optSearchOption=5, levels=1)
mv_hwa_down=MAnalyse(super_hwa_down, multi=true, delta=my_init_tr, chroma=true, mt=false, optSearchOption=5, levels=1)
mv_hwa_left=MAnalyse(super_hwa_left, multi=true, delta=my_init_tr, chroma=true, mt=false, optSearchOption=5, levels=1)
mv_hwa_right=MAnalyse(super_hwa_right, multi=true, delta=my_init_tr, chroma=true, mt=false, optSearchOption=5, levels=1)

dg_center=MDegrainN(src, super_hwa_center, mv_hwa_center, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, thSADA_a=my_thSADA_a, thSADA_b=my_thSADA_b, mt=my_MT, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

dg_up=MDegrainN(src_up, super_hwa_up, mv_hwa_up, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, thSADA_a=my_thSADA_a, thSADA_b=my_thSADA_b, mt=my_MT, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

dg_down=MDegrainN(src_down, super_hwa_down, mv_hwa_down, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, thSADA_a=my_thSADA_a, thSADA_b=my_thSADA_b, mt=my_MT, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

dg_left=MDegrainN(src_left, super_hwa_left, mv_hwa_left, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, thSADA_a=my_thSADA_a, thSADA_b=my_thSADA_b, mt=my_MT, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

dg_right=MDegrainN(src_right, super_hwa_right, mv_hwa_right, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, thSADA_a=my_thSADA_a, thSADA_b=my_thSADA_b, mt=my_MT, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)


#move shifted back
dg_up=Crop(dg_up, 0,0,dg_up.width-0,dg_up.height-shift_val).AddBorders(0,shift_val,0,0)
dg_down=Crop(dg_down, 0,shift_val,dg_down.width-0,dg_down.height-shift_val).AddBorders(0,0,0,shift_val)
dg_left=Crop(dg_left, 0,0,dg_left.width-shift_val,dg_left.height-0).AddBorders(shift_val,0,0,0)
dg_right=Crop(dg_right, shift_val,0,dg_left.width-shift_val,dg_left.height-0).AddBorders(0,0,shift_val,0)


intrl=Interleave(dg_center, dg_up, dg_down, dg_left, dg_right)
vstt=vsTTempSmooth(intrl, ythresh=200, uthresh=200, vthresh=200, pmode=1, maxr=2)
SelectEvery(vstt, 5,2)

Crop(16,16,width-32, height-32)

Prefetch(..)


Using 5x 'overlapped' processing fixes some small search errors and gives lower overall noise level. In comparison with the 'center' output clip. But still have some general search errors in comparison with onCPU MAnalyse with 'max' settings. It is still not tested as 'prefilter/1st generation' processing in 2 or more generations of MVs refining.

" see all those news in SMDegrain soon"

Maybe the only small and still important features like second input to MAnalyse and auto-thSAD to MDegrain may be ported to yet another 'simple addition to 2.7.45 pinterf version' as mostly safe from bugs changes. And Dogway may test it in the SMDegrain. I hope pinterf may return back in the 2023 and I can ask about making port of some very limited pack of simple new features to make some 'official-post-2.7.45' build of mvtools.

[DTL]

New release https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.25

Added non-linear Median-like MV filtering mode in addition to linear low-pass filtering to MDegrainN.

New params to MDegrainN:
MVMedF (integer) - default 0 (disabled). Temporal radius of median filtering of temporal MVs sequence. Valid range - from 1 to about 1/3 of tr-value used.
MVMedF_em (integer) - default 0. Edges of MV temporal vector processing modes. Mode 0 - copy non-filtered MVs from input. Mode 1 - invalidate non-filtered frames MVs to save from possible blending of bad MVs. Number of non-filtered frames equal to MVMdeF value.
MVMedF_cm (integer) - default 0. MVs coordinates processing mode. Mode 0 - separated x,y vectors median filtering. Mode 1 - using length of difference vector as dissimilarity metric.

Example:

MDegrainN(last,super, multi_vec, tr, thSAD=135, thSAD2=120, mt=false, wpow=4, thSCD1=450, thSADA_a=my_thA, thSADA_b=my_thB, \
adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, MVMedF=2, MVMedF_cm=0, MVMedF_em=0, IntOvlp=3)


For any used MV filtering mode thMVLPFCorr must be non-zero (to use filtered MVs with coordinates difference from input MVs below this value).
Both linear and non-linear MVs filtering may be enabled in any combination. First executed non-linear filtering and next linear.
MVMedF_em=1 may be used for possible higher quality processing (non-filtered MVs/blocks at the edges of tr excluded from blending). But it eats MVMedF frames from total tr-pool and decreases possible denoise level (so to keep same max denoise level with MVMedF_em=1 tr need to be tr+MVMedF).

MVMedF_cm=1 may produce more shifted areas of moving objects - it is subject of testing and may be fix possible in next releases.

Expected working values for MVMedF temporal radius of non-linear filtering - about 1..3 (may be up to about 1/3 of the tr value used, so for MVMedF=3 recommended tr is about 6..10). Values of 3 and more not yet tested.

Non-linear filtering expected to work better in skipping false long strike MVs with lower tr in compare with linear filtering.

Test script for compare new MVs processing features with old (2022 ?) source from post https://forum.doom9.org/showthread.php?p=1974040#post1974040 :

LoadPlugin("mvtools2_260923.dll")
LoadPlugin("ffms2.dll")

FFmpegSource2("test_org.mkv")

examp=FFMpegSource2("test_enc_thSAD200.mkv").Crop(250,200,500,500).ConvertToYUV420(matrix="Rec709").ConvertBits(8).Subtitle("test_enc")

ConvertToYUV420(matrix="Rec709")

Crop(250,200,500,500)

noproc=last.Subtitle("src")

super_std=MSuper(mt=false, pel=2)

tr=14

my_thA = 1.3
my_thB = 30

multi_vec=MAnalyse(super_std, blksize=8, multi=true, search=3, temporal=false, trymany=true, searchparam=2, chroma=true, delta=tr, truemotion=false,\
pzero=10, pnew=10, pglobal=10, levels=0, mt=false, overlap=4)
new_m2=MDegrainN(last,super_std, multi_vec, tr, thSAD=150, thSAD2=140, thSADA_a=my_thA, thSADA_b=my_thB, mt=false, wpow=4, thSCD1=500, \
adjSADzeromv=0.7, adjSADcohmv=0.7, thCohMV=6, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, MVMedF=2, MVMedF_cm=0, MVMedF_em=1,\
IntOvlp=0).Subtitle("new_m2")
old=MDegrainN(last,super_std, multi_vec, tr, thSAD=200, thSAD2=190, mt=false,IntOvlp=0).Subtitle("old_thSAD200")
ConvertBits(8)

Interleave(examp, new_m2, old, noproc)
Sharpen(0.5)

SincLin2Resize(width*2, height*2)

Prefetch(2)


155 (src) frame of interleaved sequence compare https://imgsli.com/MjA5OTQx/1/3

[DTL]

New release - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.26

Added new MVF_fm param to MDegrainN. Fixed MV filtering in the non-YUV-combined processing modes.

MVF_fm (integer), default=0 . Blocks failing mode at the process of MVs filtering. Mode 0 (default) - pass blocks with failed SAD re-check unchanged to blending engine. Mode 1 - fail (invalidate to blending) blocks with failed SAD re-check after filtered MVs coordinates.

In MVF_fm=1 mode it saves more blocks from blurring but it typically cause degradation of denoising at these areas. So cause uneven denosie over total frame area. May be more visible when compare static frames.

[DTL]

Some strategic announcement about fully hardware multi-generation MVs refining for noised sources.

I made some onCPU tech tests of multi-generation refining in MDegrainN (using simple ESA search algorithm) - even with a search radius of 4 it is much slower. Though still faster in comparison with script-based refining (and uses about 2 times less RAM). It is expected to be somehow faster after most possible optimizations with SIMD but not very much possible.

So as hardware MVs search ASIC typically shows significant underload - it is possible to put MVs refining into hardware accelerator (new modes for MAnalyse). Major part is development of MDegrainN simple or most featured processing as Compute Shader so it can be dispatched in HWA without downloading current generation MVs for external processing. It not breaks logic of MAnalyse in AVS filterchain - it still outputs single MVs clip with refined MVs completely in the hardware accelerator using hardware MVs search ASIC and hardware universal shaders dispatch units to dispatch SAD and MDegrainN shader to provide filtered frames to next generations of MVs refining by same hardware MVs search ASIC. For boards with 2 or more NVENC ASICs I hope drivers are smart enough to create a full filterchain inside accelerator board without download-upload resources between degrain and MVs search stages and spread load over all available ASICs onboard.

[tormento]

New release
Maybe this (https://devblogs.microsoft.com/pix/pix-2403/) can be useful.

[DTL]

New release https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.27

Added more 'area' predictors to MAnalyse. Extended optPredictorType to -1,-2, -3.

Added AreaMode MVs refining to MAnalyse.
New params:
AreaMode (integer), valid values 0,1,2,3,4.
0 - disabled
1 - x5 total positions checks (center + 4 diagonal offsets of +-1)
2 - x9 total positions checks (center + 8 diagonal offsets of +-1 and +-2)
3 - x13 total positions checks (center + 12 diagonal offsets of +-1 and +-2 and +-3)
4 - x17 total positions checks (center + 16 diagonal offsets of +-1 and +-2 and +-3 and +-4)

AMdiffSAD (integer), valid values 0 and positive. Recommended range 1..1000.
Allow to add MVs absolute difference from AreaMode search to the SAD to send additional hints about block MV quality. Multiplier to the mean sum of abs MVs coordinates differences. Values about 1000 and higher totally fail the SAD value of the block.

Example of progressive film processing script with difference control from different AreaMode setting:

tr=12
my_AMDiffSAD=0
my_thSADA_a=1.2

my_intOvlp=0
my_ovlp=4

super=MSuper(last, mt=false, pel=2, hpad=8, vpad=8)

multi_vec_cpu=MAnalyse (super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, truemotion=true, overlap=my_ovlp, chroma=false, optSearchOption=1, optPredictorType=0, mt=false)
multi_vec_am5=MAnalyse (super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, truemotion=true, overlap=my_ovlp, chroma=false, optSearchOption=1, optPredictorType=0, mt=false, AreaMode=1, AMdiffSAD=my_AMDiffSAD)
multi_vec_am9=MAnalyse (super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, truemotion=true, overlap=my_ovlp, chroma=false, optSearchOption=1, optPredictorType=0, mt=false, AreaMode=2, AMdiffSAD=my_AMDiffSAD)
multi_vec_am13=MAnalyse (super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, truemotion=true, overlap=my_ovlp, chroma=false, optSearchOption=1, optPredictorType=0, mt=false, AreaMode=3, AMdiffSAD=my_AMDiffSAD)

ma_cpu=MDegrainN(last,super, multi_vec_cpu, tr, thSADA_a=my_thSADA_a, thSADA_b=50, mt=false, wpow=4, adjSADzeromv=0.8, adjSADcohmv=0.8, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_intOvlp).Subtitle("ma_cpu")
ma_cpu_am5=MDegrainN(last,super, multi_vec_am5, tr, thSADA_a=my_thSADA_a, thSADA_b=50, mt=false, wpow=4, adjSADzeromv=0.8, adjSADcohmv=0.8, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_intOvlp).Subtitle("ma_am5")
ma_cpu_am9=MDegrainN(last,super, multi_vec_am9, tr, thSADA_a=my_thSADA_a, thSADA_b=50, mt=false, wpow=4, adjSADzeromv=0.8, adjSADcohmv=0.8, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_intOvlp).Subtitle("ma_am9")
ma_cpu_am13=MDegrainN(last,super, multi_vec_am13, tr, thSADA_a=my_thSADA_a, thSADA_b=50, mt=false, wpow=4, adjSADzeromv=0.8, adjSADcohmv=0.8, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_intOvlp).Subtitle("ma_am13")

Interleave(ma_cpu, ma_cpu_am5, ma_cpu, ma_cpu_am9, ma_cpu, ma_cpu_am13, Subtract(ma_cpu, ma_cpu_am13).Levels(100,1, 140, 0,255))
Sharpen(1.0)


The benefit from more predictors added is very small but may be tested. They are median (mode ?) of the surround block predictors from previous levels of search. Performance cost is not big (in single predictor refining mode with trymany=false).

The performance cost of AreaMode search is great - it is +4, +8, +12, +16 new full searches runs around current block position and next computing of median (mode ?) of the resulted MVs vector to create output MV (SAD is the max of the selected best dx and dy MVs found). AreaMode=3 (12 additional block searches) runs about 5 times slower in comparison with standard single search per block at i5-9600K CPU.
https://i.postimg.cc/qMkJ1BHn/AM-drawing01.png

Dark film scene compare (gamma=2.0 added to show darks better) https://imgsli.com/MjUxNjY3

Also this new level of processing generates more data for analysis of 'quality of MV' and currently a simple idea is used to additionally signal to the denoising engine: If MVs of the small shifted positions around the current block are not coherent - it means MV estimation may be unstable. So the absolute difference of the MVs coordinates (averaged to the number of MVs in the search pool to make no (less ?) dependence of the AreaMode used) multiplied to the AMdiffSAD param may be added to block SAD. So this block will take less degrain weight in MDegrain. Users can try to control the effect of this setting using MShow for average SAD per frame (showsad=true).

For the areas of stable MVs enabling AreaMode makes close to no difference and great performance hit. For the areas of unstable MVs it adds some more quality in denoising (may be good visible at 600..800% crops). x264 encoding bitrate at fixed CRF=18 is also a bit lower (about 3% at some quick test encodings). In the future I hope to add close to this idea for the DX12 accelerated ME. So the ME ASIC may be finally good loaded with useful work. But I still not yet fully restore my development environment (so currently no DX12-ME build available) and also may need to ask Microsoft support how to fast shift loaded resource in the accelerator to send a queue of searches to ASIC in 1 job list instead of uploading lots of shifted copies of the frames to make searches with shifted block tessellation grid.

The AreaMode level is located over standard levels of search in MAnalyse and compatible with any old mode but currently looks like work only with block size 8x8 because of experimental release not fully debugged. In the future it is planned to move into -e.XX builds expected to be more stable in comparison with -a.XX builds.

Also some observation: enabling truemotion=true also significantly helps to MVs stability in the low contrast low detailed noised areas recommended to be enabled in the high quality use cases.

Addition: Current ideas on performance optimization: Limit AreaMode search to full-pel level only (or to some non-finest level depending on current pel-setting). But the full search algorithm still can fail to 'best SAD' and not really best MV while checking lots of predictors (starting from zero predictor). So for using full search with AreaMode down to full-pel level and only sub-sample refining it may be added some more PredictorsTypes (like optPredictorType=5 or something else). Or as I long time thinking it is better to make many control params of MAnalyse arrays instead of single value for all levels. With arrays user can set much more flexible performance/quality balance on each level. Currently MAnalyse only have separate param like persearch to set different search radius at the sub-pel levels. But it can be expanded to many other search params to adjust performance/quality at user side and not hardcode into MAnalyse as 'hard presets'. But I still not have experience in arrays params for AVS filters. So current more easy ways is to add more params like AMll (for AreaMode level limit) and maybe pelPT (for pel PredictorType selection for fixed sets of predictors used at sub-sample levels of search) and maybe peltrymany (to select usage of refining of all predictors or best only at the sub-sample levels). So user can select more quality search at the levels down to full-sample precision (fast enough) and select only limited search at the slowest pel=2 and pel=4 levels for better performance.

[guest]

New release https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.27

I hate to say it DTL, but any build after Release_2.7.46_e.03, causes an error in RipBot264 "Cannot render the file" when previewing script in AVSMeter.

With either DX12 or noDX12, build .03 had AVX options.

[DTL]

Oh - a.XX builds are the most featured but unstable for many blocksizes/bitdepths. Typically tested only with YV12 and 8x8 blocksize. So if it crash even with YV12 and 8x8 blocksize - write at least frame size (or better full script) so I can try to look in the debugger. The e.03 build only have very small number of new features after 2.7.45 added but expected to be most stable and support all bitdepths/blocksizes as in 2.7.45.

I plan to add AreaMode search mode into e.XX builds but some time later after finishing most of its settings.

With the very limited developer resources at the residuals of current civilization the new features tested and debugged modes are quickly shrinks to the very limited and mostly used format like YV12 and 8x8 blocksize (like at the very beginning of AVS and mvtools). Though I understand the better quality of MVs is expected with at least dual-blocksize processing like first search with blocksize of 16x16 and next is refine to 8x8 with MRecalculate. So most of new controls to MAnalyse need to be added to MRecalculate too.

It may be also natural limiting to AreaMode finest level (also different area-size) - like first search MAnalyse(AreaMode=3, blksize=16) and next refine with MRecalculate(AreaMode=1, blksize=8) for better performance and maybe quality too because of larger blocksize at the first search.

[guest]

Oh - a.XX builds are the most featured but unstable for many blocksizes/bitdepths.

I will double check the script I used, but I generally don't have any blocksize settings, and don't know what the video I'm using is, either.

Will report back....