New DirectX 12 Video APIs - Page 12

anton_foy · 8th March 2023, 12:29

Quote:

Originally Posted by DTL

Some important note for multi-generation MVs refinement: The thSAD for MDegrain need to be significantly reduced after 1st generation of MAnalyse using first generation of MDegrain output. Because SAD of mostly cleaned 'current' block with input noised block become about 2time lower. So thSAD for intermediate generations and last output MDegrain need to be reduced to about 0.5 of initial.

So better multi-generation MVs refinement is some like:

Code:

init_thSAD=400

s1=MSuper()
mv1 = MAnalyse(s1)
dg1 = MDegrain(s1, mv1, thSAD=init_thSAD)

1stgen_thSAD = (int)(init_thSAD/1.8) # divisor - subject to Zopti refine ?

s2=MSuper(dg1)
mv2 = MAnalyse(s2, SuperCurrent=s1) # or (s1, SuperCurrent=s2) - may be not visible difference
dg2=MDegrain(s1, mv2, thSAD=1stgen_thSAD)

Also it was found enabling trymany=true in MAnalyse while good refining zero MVs also may add some significantly bad MVs. So it is planned to add flags for predictors used in trymany mode to skip possibly bad predictors and to make performance visibly better.

Love those updates! Seems logical with lowering the thSad in the next step. Does this work using DX12 me?

DTL · 8th March 2023, 13:14

Yes - MAnalyse with optional SuperCurrent can be used with any optSearchOption value (so including hardware search options). I even think of using >1 HWacc in the system for better performance in pipelined way. So first HWacc making initial analysis and second make refining step.

Later we will have many cheap secondhand old HWaccs capable of DX12-ME so it may be tested. Currently you can try accnum different for MAnalyses:

Code:

init_thSAD=400

s1=MSuper()
mv1 = MAnalyse(s1, optSearchOption=5, accnum=1) # use first DX12-ME accelerator in system or accnum=0 ? need testing
dg1 = MDegrain(s1, mv1, thSAD=init_thSAD)

1stgen_thSAD = (int)(init_thSAD/1.8) # divisor - subject to Zopti refine ?

s2=MSuper(dg1)
mv2 = MAnalyse(s2, SuperCurrent=s1, optSearchOption=5, accnum=2) # use second DX12-ME accelerator in system or accnum=1 - need testing 
dg2=MDegrain(s1, mv2, thSAD=1stgen_thSAD)

Currently using 2 MAnalyse with single HWacc will drop performance about 2 times.

Also may be combination of 1 external PCI-board DX12-ME acc and build-in into CPU may be tested where avaiable.

Also as I read some NVIDIA boards/chips have >1 MPEG encoder ASIC (?) so may expose >1 full-speed DX12-ME interfaces for applications.

At https://developer.nvidia.com/video-e...ort-matrix-new
# OF CHIPS

# OF NVENC /CHIP

Total # of NVENC

So GeForce GTX 965M > 980M / 980MX Maxwell (2nd Gen) may have 2 full-speed DX12-ME interfaces ?
Also GeForce GTX 960 Ti / 970 / 980 , GeForce GTX 980 Ti , GeForce GTX Titan X
GeForce GTX 1070M / 1080M , GeForce GTX 1070 / 1070Ti, GeForce GTX 1080 , GeForce GTX 1080 Ti, GeForce GTX Titan X / Titan Xp

Same is GeForce RTX 4080 Laptop , GeForce RTX 4080 16GB , GeForce RTX 4090 Laptop , GeForce RTX 4090 - but much more expensive.

Also Titan V - 3 NVENC.

Dogway have GTX 1070? May be good to try to ask for testing 1 vs 2 MAnalyse performance (also accepting different accnum values >0 or >1).

Addition: I not sure if several MPEG encoder ASICs located in single physical board will be switched as different Direct3D12 devices with 'accnum' param. May be environment will auto-spread motion estimation tasks if single board have several task dispatch resources avaialble. So 2-NVENC boards may be simply allow to run 2 MAnalyse with about equal speed with default accum=0.

DTL · 8th March 2023, 20:26

New release: https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.20

Fixed possible bug with trymany in MAnalyse.
Added trymany into optPredictorType=1 mode (zero, global and median predictors only).

Added partial fix for 4:2:x formats processing chroma shift issue for MAnalyse, MDegrainN, MCompensate (may also MRecalculate). With the curernt pel-precision from MSuper.

The multi-generations MVs refining looks like also work very visibly against blurring for complex motion like facial animation.

Cleaned from MShow processing script:

Code:

my_DMFlags=1
my_thSAD=300
my_thSAD2=250

my_thSAD_mg=150
my_thSAD2_mg=100

my_thSCD=500

my_global=true
my_pzero=10
my_pnew=10
my_pglobal=10

my_pel=2
my_trymany=true

my_oPT=1

tr=6
super=MSuper(last,chroma=true, mt=false, pel=my_pel)

multi_vec=MAnalyse(super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g1=MDegrainN(super, multi_vec, tr, thSAD=my_thSAD, thSAD2=my_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen1=g1

super_g1=MSuper(gen1,chroma=true, mt=false, pel=my_pel)
multi_vec_g2=MAnalyse(super_g1, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)

g2=MDegrainN(super, multi_vec_g2, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen2=g2

super_g2=MSuper(gen2,chroma=true, mt=false, pel=my_pel)
multi_vec_g3=MAnalyse(super_g2, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g3=MDegrainN(super, multi_vec_g3, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen3=g3

super_g3=MSuper(gen3,chroma=true, mt=false, pel=my_pel)
multi_vec_g4=MAnalyse(super_g3, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g4=MDegrainN(super, multi_vec_g4, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen4=g4

super_g4=MSuper(gen4,chroma=true, mt=false, pel=my_pel)
multi_vec_g5=MAnalyse(super_g4, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, trymany=my_trymany, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g5=MDegrainN(super, multi_vec_g5, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

gen5=g5

super_g5=MSuper(gen5,chroma=true, mt=false, pel=my_pel)
multi_vec_g6=MAnalyse(super_g5, SuperCurrent=super, multi=true, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, mt=false, optSearchOption=1, truemotion=false,  pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=my_global, levels=4, DMFlags=my_DMFlags, optPredictorType=my_oPT)
g6=MDegrainN(super, multi_vec_g6, tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3)

return Interleave(g6.Subtitle("g6"), g2.Subtitle("g2"), g1.Subtitle("g1"))

Generally gen2 is already much sharper at motion in compare with gen1 (standard MDegrainN). Gen6 sometime look a bit better and sometime more blurry. May be some average good number of generations is between 2 and 6 (or some detail-restoration processing may be added to regain details from high-gen if that frame (area of frame) is sharper).

Frames g1, g2 and g6 2x enlarged with BSpline:

It was non-field separated interlaced so 2 fields present.

May be somehow this many calls to MSuper/MAnalyse/MDegrainN for each generation of MVs refining can be compacted to some AVS function and make script smaller.

imgsli comparisons:
https://imgsli.com/MTYwODgx
https://imgsli.com/MTYwODgw

DTL · 9th March 2023, 22:57

Real working script with both accelerator and CPU search and refining functions. For 1920x1080i input.

Code:

# Input plugins
LoadPlugin("ffms2.dll")
LoadPlugin("mvtools2.dll")

SetFilterMTMode("DEFAULT_MT_MODE", 3)

my_thSAD=260
my_thSAD2=240

my_thSAD_mg=130
my_thSAD2_mg=120

my_thSCD=500

my_pzero=10
my_pnew=10
my_pglobal=10

my_pel=2
my_thCohMV=5 # 5..8 for pel=2, 10..16 for pel=4 ?
my_trymany=true

my_oPT=1
my_overlap=0
my_IntOvlp=3
my_searchparam=2

my_MPBNumIt=2

my_init_tr=12
my_refine_tr=12

Function RefineMV(clip mvclip, clip super_ref, clip src, int _thSAD, int _thSAD2, int in_tr, int refine_tr, int my_thSCD, int my_pel, bool my_trymany, int my_pnew, int my_pzero, int my_pglobal, \
int my_oPT, int my_overlap, int my_searchparam, int my_IntOvlp, int my_thCohMV)
{
 g_next=MDegrainN(src, super_ref, mvclip, in_tr, thSAD=_thSAD, thSAD2=_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=my_thCohMV, \
MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp)
 super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel)
 return MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=refine_tr, search=3, searchparam=my_searchparam, trymany=my_trymany, overlap=my_overlap, chroma=true, mt=false,\
 optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=true, optPredictorType=my_oPT)
}

Function RefineMV_HW(clip mvclip, clip super_ref, clip src, int _thSAD, int _thSAD2, int in_tr, int refine_tr, int my_thSCD, int my_pel, int my_thCohMV)
{
 g_next=MDegrainN(src, super_ref, mvclip, in_tr, thSAD=_thSAD, thSAD2=_thSAD2, mt=false, wpow=4, thSCD1=my_thSCD, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=my_thCohMV, \
MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3, UseSubShift=1)
 super_g_next=MSuper(g_next,chroma=true, mt=false, pel=my_pel, levels=1, pelrefine=false)
 return MAnalyse(super_g_next, SuperCurrent=super_ref, multi=true, delta=refine_tr, chroma=true, mt=false, optSearchOption=5, levels=1)
}


FFmpegSource2("1920x1080i.mp4")

AddBorders(0,0,0,72)

noproc=last

SeparateFields()

super_hwa=MSuper(last, mt=false, chroma=true, pel=my_pel, hpad=8, vpad=8, levels=1, pelrefine=false)
super_cpu=MSuper(last, mt=false, chroma=true, pel=my_pel, hpad=8, vpad=8, levels=0, pelrefine=true)

multi_vec_hwa=MAnalyse(super_hwa, multi=true, blksize=8, delta=my_init_tr, overlap=0, chroma=true, optSearchOption=5, mt=false, levels=1)
multi_vec_cpu=MAnalyse(super_cpu, multi=true, delta=my_init_tr, search=3, searchparam=my_searchparam, trymany=my_trymany, overlap=my_overlap, chroma=true, mt=false, \
optSearchOption=1, truemotion=false, pnew=my_pnew, pzero=my_pzero, pglobal=my_pglobal, global=true, optPredictorType=my_oPT)

multi_vec_cpu2=RefineMV(multi_vec_cpu, super_cpu, last, my_thSAD, my_thSAD2, my_init_tr, my_refine_tr, my_thSCD, my_pel, my_trymany, my_pnew, my_pzero, my_pglobal, my_oPT, \
my_overlap, my_searchparam, my_IntOvlp, my_thCohMV)
multi_vec_hybr2=RefineMV(multi_vec_hwa, super_cpu, last, my_thSAD, my_thSAD2, my_init_tr, my_refine_tr, my_thSCD, my_pel, my_trymany, my_pnew, my_pzero, my_pglobal, my_oPT, \
my_overlap, my_searchparam, my_IntOvlp, my_thCohMV)
multi_vec_hwa2=RefineMV_HW(multi_vec_hwa, super_hwa, last, my_thSAD, my_thSAD2, my_init_tr, my_refine_tr, my_thSCD, my_pel, my_thCohMV)

cpu2=MDegrainN(last,super_cpu, multi_vec_cpu2, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp, MPBthSub=5, MPBthAdd=20, MPBNumIt=my_MPBNumIt, \
MPB_SPCsub=0.5, MPB_SPCadd=1.5, MPBthIVS=2200, showIVSmask=false).Weave().Subtitle("cpu2")
hwa2=MDegrainN(last,super_hwa, multi_vec_hwa2, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp, MPBthSub=5, MPBthAdd=20, MPBNumIt=my_MPBNumIt, \
MPB_SPCsub=0.5, MPB_SPCadd=1.5, MPBthIVS=2200, showIVSmask=false).Weave().Subtitle("hwa2")
hybr2=MDegrainN(last,super_hwa, multi_vec_hybr2, my_refine_tr, thSAD=my_thSAD_mg, thSAD2=my_thSAD2_mg, mt=false, wpow=4, UseSubShift=1, thSCD1=my_thSCD, adjSADzeromv=0.7, \
adjSADcohmv=0.7, thCohMV=my_thCohMV, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_IntOvlp, MPBthSub=5, MPBthAdd=20, MPBNumIt=my_MPBNumIt, \
MPB_SPCsub=0.5, MPB_SPCadd=1.5, MPBthIVS=2200, showIVSmask=false).Weave().Subtitle("hybr2")

Interleave(cpu2, hybr2, hwa2, noproc.Subtitle("src"))

#last=hybr2

Crop(0,0,1920,1080)

Prefetch(6)

Examples of both accelerator and CPU search and refining and hybrid mode (accelerator first search and CPU refining). Quality onCPU is a bit better. Full CPU search and refine at i5-9600K and 1920x1080i frame run at about 0.28 fps.
Hybrid mode with GTX1060 and i5-9600K CPU run at about 1.24 fps (pel=4) and 1.75fps (pel=2). Quality is close to full CPU search.
Full accelerator search and refine run only a bit faster (about 1.3 fps with pel=4) and quality is a bit lower of hybrid mode at some scenes.

DTL · 15th March 2023, 18:39

New version: https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.21
Added new processing mode to MDegrainN: MEL (Most Equal Looking) search mode for TTH (Temporal Thresholding).
New params to MDegrainN:

pmode=0 (default) - standard blending, pmode=1 - MEL search and TTH only.

TTH_DMFlags - dismetric flags for estimating blocks difference at TTH compare. Flags 0x01 to 0x20 valid (except 0x08).

TTH_thUPD (0 default, additional thresholding disabled, 100% linear mode, must be >0 for pmode=1) - integer threshold for selection: keep output old in pmode=0 or 'best' in pmode=1 block from memory or update block in memory and output new block. Typical working values expected to be significantly below thSAD (like thSAD/3.. thSAD/4 and less). Startng from 0. 0 mean no blocks from memory used (standard MDegrainN mode - FIR filter).

TTH_chroma - use chroma in TTH dismetric analysis (slower, better quality) or not (faster).

Fixed performance issue with double processing of chroma planes in combined YUV processing with no overlap.

Current testscript:

Code:

tr=10
super=MSuper(last, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true)
multi_vec=MAnalyse(super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, overlap=0, optSearchOption=1, optPredictorType=0, chroma=false, mt=false)
ref=MDegrainN(last,super, multi_vec, tr, thSAD=185, thSAD2=170, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, IntOvlp=3)

super2=MSuper(ref, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true)
MDegrainN(ref,super2, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, thSCD1=350, pmode=1, TTH_thUPD=100, IntOvlp=3)

TTH_thUPD may be also enabled in 'standard' blending modes (pmode=0 (default)) too (both overlap and no overlap combined YUV processing). It is (much) faster but may provide somehow lower quality. Currently no motion block tracking available so it is mostly effective for completely static blocks only. Some limited motion tracking expected to be in some future versions.

Complexity of analysis in pmode=1 currently is ~tr^2 so it may use separate tr value (and mvclip created with lower tr value). Quality expected to be ~tr value (probability to found most commonly looking block in the total tr-pool). Param thSAD in pmode=1 also controls initial block skipping when accumulating blocks in analysis pool.

TTH_thUPD is the main param to adjust - the higher its value - the more noise blocks are skipped but too high value may cause 'hanging' blocks visible or motion quality degradation. Setting too high thSAD in pmode=1 also may cause more artifacts.

pmode=1 expected to be 'final cleaning' after initial MDegrainN (also must use new super clip with pre-denoised frames) and if highest quality required. For general everyday encodings may be enough to play with TTH_thUPD param in standard pmode=0.

Last MDegrainN with pmode=1 may or may not use refined mv-clip (for best results best refined mvclip is recommended).

pmode=1 not blend at all - so no degrade details quality with any thSAD. It only additional way to select 'best' looking block in current tr-scope and duplicate it in output frames until visual difference with current frame block is below threshold.

TTH_thUPD may be enabled in any MDegraiN in processing script (in MVs refining and final degrain and final cleanup).
TTH_DMFlags may set any avaialable dismetric for visual difference analysis (SAD - faster, SSIM and VIF - slower) at any MDegrainN with enabled TTH_thUPD or pmode=1.

DTL · 30th March 2023, 11:04

Some morning quicky implementation of this year idea about noise bitrate estimation to check the degrain quality.

Release 30.03.2023 - https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.22

Added computing and displaying of residual noise bits count per frame to MCompensate.
Compute sum of log2 of the samples absolute difference between source and motion compensated output frame of MCompensate. For complete static frame sequence RNB=0 bits/frame. For noise bitrate per second - value should be multiplied to frame rate.

New param to MCompensate: showRNB (default = false).

Usage example:
super=MSuper()
mv=MAnalyse(super)
MCompensate(super, mv, showRNB=true)

Currently only for 8bit sources. Need to offset processing function to templated for HBD support. Can process Y only input clip or YUV/RGB (3 planes present). For >1 planes the sum of all planes is displayed.

Computing part:

Code:

    for (int y = 0; y < nHeight; y++)
    {
      uint8_t* pDstFrame = pDst[0] + nDstPitches[0] * y;
      uint8_t* pSrcFrame = (uint8_t*)pSrc[0] + nSrcPitches[0] * y + nOffset[0];
      for (int x = 0; x < nWidth; x++)
      {
        iSumNzBits += 32 - __lzcnt(SADABS((int)pSrcFrame[x] - (int)pDstFrame[x]));
      }
    }

Not applicable to float32 samples directly (need convert to some finite precision integer first <32bit).

Usage example to measure denoise process:

Code:

SeparateFields()

fields_orig=last

tr=3

super=MSuper(last, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true)
multi_vec=MAnalyse(super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, overlap=0, optSearchOption=1, optPredictorType=0, chroma=false, mt=false, DMFlags=1)
ref=MDegrainN(last,super, multi_vec, tr, thSAD=185, thSAD2=170, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, IntOvlp=3)

super2=MSuper(ref, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true)
MDegrainN(ref,super2, multi_vec, tr, thSAD=350, thSAD2=340, mt=false, thSCD1=350, pmode=1, TTH_thUPD=100, IntOvlp=3)

super_ref=MSuper(ref)
mv_ref=MAnalyse(super_ref)
rnb_den_ref=MCompensate(super_ref, mv_ref, showRNB=true)

super2=MSuper()
mv2=MAnalyse(super2)
rnb_den=MCompensate(super2, mv2, showRNB=true)

super_orig=MSuper(fields_orig)
mv_orig=MAnalyse(super_orig)
rnb_orig=MCompensate(super_orig, mv_orig, showRNB=true)

StackHorizontal(rnb_den, rnb_den_ref,  rnb_orig)

Weave()

Output sample frame

Yes - the fileds are blended not very nice. It shows how for static areas the second MDegraiN(pmode=1) decreases noise bitcount about 10 times. First stage denoise about 2.9 times decrease noise bitrate. Addition of secondary non-linear IIR-type filter with memory decreases nosie bitrate about 30 times from source.

Completely (100%) temporal denoised frame sequence for zero calibration is
Trim(1,1)
Loop()

TDS · 30th March 2023, 11:49

Quote:

Originally Posted by DTL

Some morning quicky implementation of this year about noise bitrate estimation to check the degrain quality.

Release 30.03.2023 - https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.22

Just to confirm, and probably a noobie question, but the 2 variants, DX12 & noDX12, are reliant on Direct-X being installed on the system (or not) ??

DTL · 30th March 2023, 14:03

If you have Win10 or later and compatible hardware you can use DX12 build. It will not load at Win7 or other without DX12 installed. If you not use DX12 search modes in MAnalyse you can safely use noDX12 build.

mastrboy · 30th March 2023, 18:54

DTL, I can't get your builds to work at all in Windows11, I have tried all 4 different .dll's in the zip file...

AVSmeter just stops at 0 frames forever, until I hit ctrl+c.

Code:

AVSMeter64.exe -o d:\test.avs

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r3973, 3.7, x86_64) (3.7.3.0)

Number of frames:                    33304
Length (hh:mm:ss.ms):         00:23:09.054
Frame width:                           960
Frame height:                          720
Framerate:                          23.976 (24000/1001)
Colorspace:                           YV12

Frame (current | last):         0 | 33303

Virtualdub gives me a cryptic memory violation message:

Code:

An out-of-bounds memory access (access violation) occurred in module 'VirtualDub64'...
...reading address FFFFFFFFFFFFFFFF.

Avspmod gives a similear error message:

Code:

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 315, in 'calling callback function'
  File "avsp.pyo", line 5136, in local_wnd_proc
WindowsError: exception: access violation reading 0xFFFFFFFFFFFFFFFF

I have none of these issues with https://github.com/pinterf/mvtools/releases

I also have no idea how to troubleshoot this other than give you some information about my system and hope you have any idea of what is wrong:
AviSynth+ 3.7.3 (r3973, 3.7, x86_64)
Windows 11 22H2 (22621.1413)

Avisynth script I tested with:

Code:

FFVideoSource("test.mkv")
SMDegrain(tr=3, thSAD=400, RefineMotion=false, contrasharp=false, plane=4, prefilter=0, chroma=true)

DTL · 30th March 2023, 19:41

Unfortunately my builds may be not compatible with many old scripts (using no-default block size of 16x16 and may be some more not tested options). So it still pre-release demos of some features and mostly tested at the examples scripts provided here and typically block size of 8x8. I even make somehow changed QTGMC to use with my builds when I tested deinterlacing.

So it is no good to put this .dll in 'common' folder and recommended to load with LoadPlugin() from current working folder. It is expected may be in some years (in beginning of 2024 it is expected great all planet celebration of 20 years for mvtools) we will have some features ported to 'more official' pinterf or may be other programmer capable to test and bugfix most of supported modes of mvtools. But it still not happen. I going to make some e-table (may be google web docs ?) of all new features and ideas accumulated and partially implemented for post-2.7.45 version with current 'status' and other data for analyse and creating list of mostly important features to port/bugfix.

Also I not use SMDegrain script and make my own denosie scripts based on mvtools only. So I not know what cause crash there. May be some day I will have time to attempt to install SMDegrain and check it with debugger where may be crash with that settings and if it possible to more or less fast to fix it.

For the very first possible solution it is recommended to test with block size of 8x8 (internal default for mvtools).

Though if you use SMDegrain as I read it still not support any new features of post-2.7.45 mvtools so it may be safely to use old 'stable' 2.7.45 build from pinterf. May be still many years until we will have some more stable post-2.7.45 build fully compatible with 2.7.45 processing with default new settings and Dogway will make changes to SMDegrain to use new features.

takla · 31st March 2023, 22:22

@DTL
Have you seen this
https://devblogs.microsoft.com/direc...y-sdk-1-710-0/
Is it applicable for mvtools?

DTL · 31st March 2023, 23:05

I have some strategic idea how may be make current post-2.7.45 version more compatible with old scripts and 2.7.45 build - rename all filters with adding _a to the end (like alpha-state). So it may be possible to load both 2.7.45 and post-2.7.45 mvtools in single AVS environment and only use selected filters from post-2.7.45 if required (also it may be (partially) compatible in between - super and mv clips). Now because of same naming it either not loads or may cause undefined usage of different filters from different .dlls. May be in next build.

"Is it applicable for mvtools?"

About new heaps mode - currently the performance is very few limited by textures upload and backward download of MVs and SAD data is very small in size. About sampling - currently some 'simple' sampling mode used in SAD compute shader (sort of sample(x,y) request as CPU from host RAM do (not possible 'complex 3D' sampling when texture mapping to some virtual triangle or other patch performed). So no update of sampling required and can not help in performance.

8th March 2023, 13:14	#222 \| Link
DTL Registered User Join Date: Jul 2018 Posts: 901	Yes - MAnalyse with optional SuperCurrent can be used with any optSearchOption value (so including hardware search options). I even think of using >1 HWacc in the system for better performance in pipelined way. So first HWacc making initial analysis and second make refining step. Later we will have many cheap secondhand old HWaccs capable of DX12-ME so it may be tested. Currently you can try accnum different for MAnalyses: Code: init_thSAD=400 s1=MSuper() mv1 = MAnalyse(s1, optSearchOption=5, accnum=1) # use first DX12-ME accelerator in system or accnum=0 ? need testing dg1 = MDegrain(s1, mv1, thSAD=init_thSAD) 1stgen_thSAD = (int)(init_thSAD/1.8) # divisor - subject to Zopti refine ? s2=MSuper(dg1) mv2 = MAnalyse(s2, SuperCurrent=s1, optSearchOption=5, accnum=2) # use second DX12-ME accelerator in system or accnum=1 - need testing dg2=MDegrain(s1, mv2, thSAD=1stgen_thSAD) Currently using 2 MAnalyse with single HWacc will drop performance about 2 times. Also may be combination of 1 external PCI-board DX12-ME acc and build-in into CPU may be tested where avaiable. Also as I read some NVIDIA boards/chips have >1 MPEG encoder ASIC (?) so may expose >1 full-speed DX12-ME interfaces for applications. At https://developer.nvidia.com/video-e...ort-matrix-new # OF CHIPS # OF NVENC /CHIP Total # of NVENC So GeForce GTX 965M > 980M / 980MX Maxwell (2nd Gen) may have 2 full-speed DX12-ME interfaces ? Also GeForce GTX 960 Ti / 970 / 980 , GeForce GTX 980 Ti , GeForce GTX Titan X GeForce GTX 1070M / 1080M , GeForce GTX 1070 / 1070Ti, GeForce GTX 1080 , GeForce GTX 1080 Ti, GeForce GTX Titan X / Titan Xp Same is GeForce RTX 4080 Laptop , GeForce RTX 4080 16GB , GeForce RTX 4090 Laptop , GeForce RTX 4090 - but much more expensive. Also Titan V - 3 NVENC. Dogway have GTX 1070? May be good to try to ask for testing 1 vs 2 MAnalyse performance (also accepting different accnum values >0 or >1). Addition: I not sure if several MPEG encoder ASICs located in single physical board will be switched as different Direct3D12 devices with 'accnum' param. May be environment will auto-spread motion estimation tasks if single board have several task dispatch resources avaialble. So 2-NVENC boards may be simply allow to run 2 MAnalyse with about equal speed with default accum=0. Last edited by DTL; 8th March 2023 at 14:59.

15th March 2023, 18:39	#225 \| Link
DTL Registered User Join Date: Jul 2018 Posts: 901	New version: https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.21 Added new processing mode to MDegrainN: MEL (Most Equal Looking) search mode for TTH (Temporal Thresholding). New params to MDegrainN: pmode=0 (default) - standard blending, pmode=1 - MEL search and TTH only. TTH_DMFlags - dismetric flags for estimating blocks difference at TTH compare. Flags 0x01 to 0x20 valid (except 0x08). TTH_thUPD (0 default, additional thresholding disabled, 100% linear mode, must be >0 for pmode=1) - integer threshold for selection: keep output old in pmode=0 or 'best' in pmode=1 block from memory or update block in memory and output new block. Typical working values expected to be significantly below thSAD (like thSAD/3.. thSAD/4 and less). Startng from 0. 0 mean no blocks from memory used (standard MDegrainN mode - FIR filter). TTH_chroma - use chroma in TTH dismetric analysis (slower, better quality) or not (faster). Fixed performance issue with double processing of chroma planes in combined YUV processing with no overlap. Current testscript: Code: tr=10 super=MSuper(last, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true) multi_vec=MAnalyse(super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, overlap=0, optSearchOption=1, optPredictorType=0, chroma=false, mt=false) ref=MDegrainN(last,super, multi_vec, tr, thSAD=185, thSAD2=170, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, IntOvlp=3) super2=MSuper(ref, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true) MDegrainN(ref,super2, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, thSCD1=350, pmode=1, TTH_thUPD=100, IntOvlp=3) TTH_thUPD may be also enabled in 'standard' blending modes (pmode=0 (default)) too (both overlap and no overlap combined YUV processing). It is (much) faster but may provide somehow lower quality. Currently no motion block tracking available so it is mostly effective for completely static blocks only. Some limited motion tracking expected to be in some future versions. Complexity of analysis in pmode=1 currently is ~tr^2 so it may use separate tr value (and mvclip created with lower tr value). Quality expected to be ~tr value (probability to found most commonly looking block in the total tr-pool). Param thSAD in pmode=1 also controls initial block skipping when accumulating blocks in analysis pool. TTH_thUPD is the main param to adjust - the higher its value - the more noise blocks are skipped but too high value may cause 'hanging' blocks visible or motion quality degradation. Setting too high thSAD in pmode=1 also may cause more artifacts. pmode=1 expected to be 'final cleaning' after initial MDegrainN (also must use new super clip with pre-denoised frames) and if highest quality required. For general everyday encodings may be enough to play with TTH_thUPD param in standard pmode=0. Last MDegrainN with pmode=1 may or may not use refined mv-clip (for best results best refined mvclip is recommended). pmode=1 not blend at all - so no degrade details quality with any thSAD. It only additional way to select 'best' looking block in current tr-scope and duplicate it in output frames until visual difference with current frame block is below threshold. TTH_thUPD may be enabled in any MDegraiN in processing script (in MVs refining and final degrain and final cleanup). TTH_DMFlags may set any avaialable dismetric for visual difference analysis (SAD - faster, SSIM and VIF - slower) at any MDegrainN with enabled TTH_thUPD or pmode=1. Last edited by DTL; 16th March 2023 at 10:25.

30th March 2023, 11:04	#226 \| Link
DTL Registered User Join Date: Jul 2018 Posts: 901	Some morning quicky implementation of this year idea about noise bitrate estimation to check the degrain quality. Release 30.03.2023 - https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.22 Added computing and displaying of residual noise bits count per frame to MCompensate. Compute sum of log2 of the samples absolute difference between source and motion compensated output frame of MCompensate. For complete static frame sequence RNB=0 bits/frame. For noise bitrate per second - value should be multiplied to frame rate. New param to MCompensate: showRNB (default = false). Usage example: super=MSuper() mv=MAnalyse(super) MCompensate(super, mv, showRNB=true) Currently only for 8bit sources. Need to offset processing function to templated for HBD support. Can process Y only input clip or YUV/RGB (3 planes present). For >1 planes the sum of all planes is displayed. Computing part: Code: for (int y = 0; y < nHeight; y++) { uint8_t* pDstFrame = pDst[0] + nDstPitches[0] * y; uint8_t* pSrcFrame = (uint8_t)pSrc[0] + nSrcPitches[0] y + nOffset[0]; for (int x = 0; x < nWidth; x++) { iSumNzBits += 32 - __lzcnt(SADABS((int)pSrcFrame[x] - (int)pDstFrame[x])); } } Not applicable to float32 samples directly (need convert to some finite precision integer first <32bit). Usage example to measure denoise process: Code: SeparateFields() fields_orig=last tr=3 super=MSuper(last, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true) multi_vec=MAnalyse(super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, overlap=0, optSearchOption=1, optPredictorType=0, chroma=false, mt=false, DMFlags=1) ref=MDegrainN(last,super, multi_vec, tr, thSAD=185, thSAD2=170, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, IntOvlp=3) super2=MSuper(ref, mt=false, chroma=true, pel=2, hpad=8, vpad=8, levels=0, pelrefine=true) MDegrainN(ref,super2, multi_vec, tr, thSAD=350, thSAD2=340, mt=false, thSCD1=350, pmode=1, TTH_thUPD=100, IntOvlp=3) super_ref=MSuper(ref) mv_ref=MAnalyse(super_ref) rnb_den_ref=MCompensate(super_ref, mv_ref, showRNB=true) super2=MSuper() mv2=MAnalyse(super2) rnb_den=MCompensate(super2, mv2, showRNB=true) super_orig=MSuper(fields_orig) mv_orig=MAnalyse(super_orig) rnb_orig=MCompensate(super_orig, mv_orig, showRNB=true) StackHorizontal(rnb_den, rnb_den_ref, rnb_orig) Weave() Output sample frame Yes - the fileds are blended not very nice. It shows how for static areas the second MDegraiN(pmode=1) decreases noise bitcount about 10 times. First stage denoise about 2.9 times decrease noise bitrate. Addition of secondary non-linear IIR-type filter with memory decreases nosie bitrate about 30 times from source. Completely (100%) temporal denoised frame sequence for zero calibration is Trim(1,1) Loop() Last edited by DTL; 30th March 2023 at 14:30.

30th March 2023, 18:54	#229 \| Link
mastrboy Registered User Join Date: Sep 2008 Posts: 363	DTL, I can't get your builds to work at all in Windows11, I have tried all 4 different .dll's in the zip file... AVSmeter just stops at 0 frames forever, until I hit ctrl+c. Code: AVSMeter64.exe -o d:\test.avs AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021 AviSynth+ 3.7.3 (r3973, 3.7, x86_64) (3.7.3.0) Number of frames: 33304 Length (hh:mm:ss.ms): 00:23:09.054 Frame width: 960 Frame height: 720 Framerate: 23.976 (24000/1001) Colorspace: YV12 Frame (current \| last): 0 \| 33303 Virtualdub gives me a cryptic memory violation message: Code: An out-of-bounds memory access (access violation) occurred in module 'VirtualDub64'... ...reading address FFFFFFFFFFFFFFFF. Avspmod gives a similear error message: Code: Traceback (most recent call last): File "_ctypes/callbacks.c", line 315, in 'calling callback function' File "avsp.pyo", line 5136, in local_wnd_proc WindowsError: exception: access violation reading 0xFFFFFFFFFFFFFFFF I have none of these issues with https://github.com/pinterf/mvtools/releases I also have no idea how to troubleshoot this other than give you some information about my system and hope you have any idea of what is wrong: AviSynth+ 3.7.3 (r3973, 3.7, x86_64) Windows 11 22H2 (22621.1413) Avisynth script I tested with: Code: FFVideoSource("test.mkv") SMDegrain(tr=3, thSAD=400, RefineMotion=false, contrasharp=false, plane=4, prefilter=0, chroma=true) __________________ (i have a tendency to drunk post) Last edited by mastrboy; 30th March 2023 at 18:56.

30th March 2023, 19:41	#230 \| Link
DTL Registered User Join Date: Jul 2018 Posts: 901	Unfortunately my builds may be not compatible with many old scripts (using no-default block size of 16x16 and may be some more not tested options). So it still pre-release demos of some features and mostly tested at the examples scripts provided here and typically block size of 8x8. I even make somehow changed QTGMC to use with my builds when I tested deinterlacing. So it is no good to put this .dll in 'common' folder and recommended to load with LoadPlugin() from current working folder. It is expected may be in some years (in beginning of 2024 it is expected great all planet celebration of 20 years for mvtools) we will have some features ported to 'more official' pinterf or may be other programmer capable to test and bugfix most of supported modes of mvtools. But it still not happen. I going to make some e-table (may be google web docs ?) of all new features and ideas accumulated and partially implemented for post-2.7.45 version with current 'status' and other data for analyse and creating list of mostly important features to port/bugfix. Also I not use SMDegrain script and make my own denosie scripts based on mvtools only. So I not know what cause crash there. May be some day I will have time to attempt to install SMDegrain and check it with debugger where may be crash with that settings and if it possible to more or less fast to fix it. For the very first possible solution it is recommended to test with block size of 8x8 (internal default for mvtools). Though if you use SMDegrain as I read it still not support any new features of post-2.7.45 mvtools so it may be safely to use old 'stable' 2.7.45 build from pinterf. May be still many years until we will have some more stable post-2.7.45 build fully compatible with 2.7.45 processing with default new settings and Dogway will make changes to SMDegrain to use new features. Last edited by DTL; 30th March 2023 at 20:14.

30th March 2023, 14:03	#228 \| Link
DTL Registered User Join Date: Jul 2018 Posts: 901	If you have Win10 or later and compatible hardware you can use DX12 build. It will not load at Win7 or other without DX12 installed. If you not use DX12 search modes in MAnalyse you can safely use noDX12 build.

31st March 2023, 22:22	#231 \| Link
takla Registered User Join Date: May 2018 Posts: 151	@DTL Have you seen this https://devblogs.microsoft.com/direc...y-sdk-1-710-0/ Is it applicable for mvtools?

31st March 2023, 23:05	#232 \| Link
DTL Registered User Join Date: Jul 2018 Posts: 901	I have some strategic idea how may be make current post-2.7.45 version more compatible with old scripts and 2.7.45 build - rename all filters with adding _a to the end (like alpha-state). So it may be possible to load both 2.7.45 and post-2.7.45 mvtools in single AVS environment and only use selected filters from post-2.7.45 if required (also it may be (partially) compatible in between - super and mv clips). Now because of same naming it either not loads or may cause undefined usage of different filters from different .dlls. May be in next build. "Is it applicable for mvtools?" About new heaps mode - currently the performance is very few limited by textures upload and backward download of MVs and SAD data is very small in size. About sampling - currently some 'simple' sampling mode used in SAD compute shader (sort of sample(x,y) request as CPU from host RAM do (not possible 'complex 3D' sampling when texture mapping to some virtual triangle or other patch performed). So no update of sampling required and can not help in performance.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode