View Single Post
Old 24th October 2022, 20:35   #174  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 1,041
New test release: https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.19
Added MPB_MVlth param to MDegrainN. Limit allowed length of MV for weight correctionby MPB processing. Can decrease possible 'ghosting' with too extreme MPB_SPCsub/add params values. Not scaled by pel currently. Recommended values - about 2..3 squared pel value. Valid working range from 0 to squared frame size (unlimited). Zero may be disable MPB weight adjusting completely.

Added reduced tr blending mode for MPB controlled currently by MPB_SPCadd > 10.

Added MPBtgtTR param to MDegrainN:
In standard MPB mode controls initial number of weights used for calculate initial blend estimation (may be 0 - only current block used).
Valid range - from 0 to tr.
In reduced weights MPB mode - controls number of ref frames (2 * tr) used for blending without any other weights adjustment by MPB.

Added MPB_DMFlags=64 flag. Uses covariance metric only. Can be used only with MDegrainN.

Added VIF (DWT- based) metric. Controlled by 0x10 (VIF-Approximation) and 0x20 (VIF-Edges) flags. 16 and 32 decimal. Full VIF (VIF-A * VIF-E) is DMFlags=16+32=48. Can be used in both MDegrainN MPB flags and MAnalyse.

Current possibly best settings for processing:
Code:
MDegrainN(last,super, multi_vec, tr, thSAD=250, thSAD2=240, thSADC=240, mt=false, wpow=4, thSCD1=400, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, 
thMVLPFCorr=50, UseSubShift=1, IntOvlp=3, MPBthSub=5, MPBthAdd=5, MPBNumIt=3, MPB_SPCadd=3, MPB_SPCsub=0.3, MPBthIVS=1500, showIVSmask=false, MPB_DMFlags=64, MPB_MVlth=8, MPBtgtTR=tr)
Example of reduced tr blending mode without MPB weights adjusting:
Code:
MDegrainN(last,super, multi_vec, tr, thSAD=250, thSAD2=240, thSADC=240, mt=false, wpow=4, thSCD1=400, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, 
thMVLPFCorr=50, UseSubShift=1, IntOvlp=3, MPBNumIt=1, MPB_SPCadd=11,  MPBthIVS=1500, showIVSmask=false, MPBtgtTR=tr-5)
Blocks with tr cut to tr-5 controlled by IVS mask.

With enough aggressive settings like MPBNumIt=3, MPB_SPCadd=3, MPB_SPCsub=0.3 it is recommended to use low enough MPB_MVlth=8 (about 2 squared pel value) or the small moving objects may start to have 'ghosts' . This 'ghosting' feature of MPG is not perfectly fixed (and even not completely debugged) so added limitation of max MV length for weight adjustment. Current idea of this ghosting: Some blocks far enough from moving objects with errors MVs got some non zero weight in total blending pool of blocks and MPB multi-pass greatly amplificated weight of such blocks as it have some details and they become visible at output. So they looks like exist in standard mvtools/MDegrainN processing but with typical low weight are invisible.

Current MPB_DMFlags=64 uses covariance only metric that is inverted to typical metrics (covariance increases with increasing blocks similarity) and do not have good max expected values to make easy inverting to align with other dis-similarity metrics so it currently implemented outside DisMetric class and uses separate processing function in MDegrainN only. Currently with enough aggressive MPB settings it looks make best result on details keeping (and even look like produce some 'sharpening' effect).

Quote:
Originally Posted by anton_foy View Post
Just a thought here, would it be possible to implement in mvtools a way to analayse and detect the grain/noise amount (using gpu?) for it to automatically and dynamically adjust tr/thsad as an option? I guess it would be a runtime thing that will slow down alot though.
Currently with progress of multi-pass blending and in the future the multi-frame multi-pass blending the importance of thSAD will be less and less important. In this processing the weights of blocks to blend are many times adjusted by many functions. So initial thSAD is for first approximations and to skip too bad blocks. Same as you see the RIFE do not have thSAD or close adjustment and process all blocks similarity automatic.

About offloading of MDegrain MPB and MFMPB to accelerator - it may be possible but in some far future. With the increasing computing complexity of 'self-adjusting blending' the penalty for upload and download result to and from accelerator may be not very big even for per-block processing. Better to upload full set of frames and download resulted frame only but it require more complex compute shader design. Also I do not have good way to debug compute shaders now with remote debug. Though with MDegrainN the host do not need DX12-ME features for run separate compute shader and may be local debug will work on much poor accelerator that exist in some way on my development host with visual studio.

Currently the MPB still not very slow with much more computing in compare with old single pass blending mode with thSAD-defined weights only because it uses very small blocks data once that is good cached to L1D cache of CPU and no more disturbs very slow host RAM. So it good benefits from multi core CPUs and fast cores even with slow enduser RAM with low memory channels and poor performance cache/memory controllers of endusers cheap CPUs.

Last edited by DTL; 26th October 2022 at 04:39.
DTL is offline   Reply With Quote