New DirectX 12 Video APIs

[DTL]

Can you post sample video and script used ?

[guest]

Can you post sample video and script used ?

Well, another day, and a different result...

.27 is now working, and all I have changed is to update FFmpeg...

[DTL]

The sad truth about complex bugs with 'memory corruption' in the still poorly protected memory in current OS/CPUs design - if some thread writes out of the buffer it may be not any visible while the overwritten memory region not used as some executable page or non-checked data bytes. It may be the main source of 'hidden bugs' accumulation - they are invisible untill damage something else important and cause crash finally. The complexity of search of these bugs is because there is no records who and when made write to damaged memory bytes. Crash happens only when damaged data read and used somewhere. Also because of close to random 4KB RAM pages location in address space - each next OS reboot and each next process run may cause different damaged memory area.

So the simple version: a.XX build with some settings cause some RAM page corruption but not all ffmpeg builds uses same memory allocation scheme so this corruption may remain hidden and not damage any important data. So ffmpeg runs without crash and maybe without image data distortion. But everything may change at any time. This is sad process of constant accumulation of overall non-stability and hard to debug the real place where it starts. There exists some automated software tools for program instrumenting for many (every ?) write operation control to help found these bugs. But they may work with C-writes and not check long SIMD data writes. And mvtools uses many hand coded asm parts and these parts may be not compatible with automated instruments for memory damaging analysis.

[tormento]

New release 2.7.46-a.27
I can't test on my Sandy Bridge, gives instruction error.

Did you compile for AVX2 only?

[DTL]

New release: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.28

Added new params to MAnalyse and MRecalculate:

AMstep (integer): default 0 (auto) or 1 and higher. Step of each 4 area search positions offset (diagonal) around center position of block. 0 auto mean scaled to 8x8 block size (auto mean 1 for 8x8 block size, 2 for 16x16 block size and so on).
AMoffset (integer): default 0. Offset from 1 to start of area positions.

New params for MAnalyse:
PTpel (integer): PredictorType at sub-sample levels of search. Default = optPredictorType.
AMpel (integer): AreaMode for sub-sample levels (level 0 for pel=1 or level 0 and level 1 for pel=2). Default = AreaMode.

New params for MRecalculate:
AreeaMode, AMdiffSAD - same as for MAnalyse.

Fixed issue with crash with block size 16x16 (and possibly others) with SIMD instructions enabled (SetMAXCPU > 'none').

Added passing of tr-value to nTrad new member of analysisdata structure to mvmulti clip from MRecalculate (to be compatible with compatibility check in new MDegrainN).

"I can't test on my Sandy Bridge, gives instruction error."

Yes - that was AVX2 only build. I added SSE2 and AVX2 in update to current release a.28 at github.

Example of test script starting with pel=1 and blocksize of 16x16 and MRecalculate to pel=2 and blocksize 8x8:

my_tr=12
my_AMDiffSAD=0
my_thSADA_a=1.2

my_intOvlp=0
my_ovlp=0
my_blksize=16
mymrecthSAD=200

super_p2=MSuper(last, mt=false, pel=2, hpad=32, vpad=32)
super_p1=MSuper(last, mt=false, pel=1, hpad=32, vpad=32)

multi_vec_cpu=MAnalyse (super_p2, multi=true, blksize=8, delta=my_tr, search=3, searchparam=2, truemotion=true, overlap=my_ovlp, chroma=false, optSearchOption=1, optPredictorType=0, mt=false)

vec_am0=MAnalyse (super_p1, multi=true, blksize=my_blksize, delta=my_tr, search=3, searchparam=2, truemotion=false, pnew=0, global=true, overlap=my_ovlp, chroma=true,\
optSearchOption=1, optPredictorType=0, mt=false, AreaMode=0, AMstep=2, AMdiffSAD=my_AMDiffSAD)
vec_am1=MAnalyse (super_p1, multi=true, blksize=my_blksize, delta=my_tr, search=3, searchparam=2, truemotion=false, pnew=0, global=true, overlap=my_ovlp, chroma=true,\
optSearchOption=1, optPredictorType=0, mt=false, AreaMode=1, AMstep=2, AMdiffSAD=my_AMDiffSAD)


multi_vec_mrec_am0=MRecalculate(super_p2, vec_am0, thSAD=mymrecthSAD, blksize=8, search=3, searchparam=4, truemotion=false, pnew=0, chroma=true, overlap=my_ovlp,\
AreaMode=0, tr=my_tr)
multi_vec_mrec_am2=MRecalculate(super_p2, vec_am1, thSAD=mymrecthSAD, blksize=8, search=3, searchparam=4, truemotion=false, pnew=0, chroma=true, overlap=my_ovlp, \
AreaMode=2, tr=my_tr)


ma_cpu=MDegrainN(last,super_p2, multi_vec_cpu, my_tr, thSADA_a=my_thSADA_a, thSADA_b=50, mt=false, wpow=4, adjSADzeromv=0.8, adjSADcohmv=0.8, thCohMV=16,\
MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_intOvlp).Subtitle("ma_cpu")
ma_cpu_mrec_am0=MDegrainN(last,super_p2, multi_vec_mrec_am0, my_tr, thSADA_a=my_thSADA_a, thSADA_b=50, mt=false, wpow=4, adjSADzeromv=0.8, adjSADcohmv=0.8,\
thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_intOvlp).Subtitle("ma_am_mrec_am0")
ma_cpu_mrec_am=MDegrainN(last,super_p2, multi_vec_mrec_am2, my_tr, thSADA_a=my_thSADA_a, thSADA_b=50, mt=false, wpow=4, adjSADzeromv=0.8, adjSADcohmv=0.8,\
thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_intOvlp).Subtitle("ma_am_mrec_am")


Interleave(ma_cpu, ma_cpu_mrec_am0, ma_cpu_mrec_am, Subtract(ma_cpu, ma_cpu_mrec_am0).Levels(100,1, 140, 0,255), Subtract(ma_cpu, ma_cpu_mrec_am).Levels(100,1, 140, 0,255))
Sharpen(1.0)


Still not any best params adjusted - just initial working to first tests.

[tormento]

Yes - that was AVX2 only build. I added SSE2 and AVX2 in update to current release a.28 at github.
:thanks:

[tormento]

Yes - that was AVX2 only build. I added SSE2 and AVX2 in update to current release a.28 at github.
I can see AVX2/SSE2 build only.

If you can, please provide AVX version too, so I can report issues if any.

[tormento]

Benchmark on a simple SMDegrain:

MVTools 2.7.46·e03 (AVX ) 7,93 fps
MVTools 2.7.46·a28 (SSE2) 7,28 fps

Then I checked some metrics between MVTools 2.7.46·e03 and MVTools 2.7.46·a28:

PSNR 52.9363
SSIM 0.9974
VMAF 96.0528

[DTL]

I updated release from 04.04.24 with DX12 and noDX12 SSE2/AVX/AVX2 builds with Visual Studio 2019.

Unfortunately a.XX versions may be really slower in compare with very close to 2.7.45 version e.03 version because of active usage of many possible dissimilarity metrics (SAD, SSIM, VIF) and runtime sub-sample shifting in MAnalyse. a.XX builds are full featured for MAnalyse but contains lots of conditional jumps. These conditional jumps disturbs out-of-order and branch-prediction units of CPUs and may make visible performance penalty.

To regain some performance you need to enable at least optSearchOption=1.

To somehow fix this new performance penalty it is planned to re-design program text of MAnalyse to templated versions of processing functions (use DMFlags dissimilarity metric control or old SAD only and UseSubShift or not). Manual copy of these options and selection of used function of startup will creates even more scary file PlaneofBlocks.cpp and more complex to future development. But this planned templated re-design only possible in some future versions. Now if testing shows the a.28 version finally start to work with typical still used complex scripts like SMDegrain and QTGMC - I can only ask Asd-g to make LLVM builds (typically somehow faster). And if LLVM compiler is more smart and AI-powered inside as we expect from still not completely died progress of civilization it may auto-optimize these conditional jumps better in comparison with simple VS2019 compiler.

[tormento]

I updated release from 04.04.24 with DX12 and noDX12 SSE2/AVX/AVX2 builds with Visual Studio 2019.
Thanks. I will try and report ASAP.

I think you should give a try to Intel Compiler. AFAIK it produces the fastest builds, even for AMD.
To regain some performance you need to enable at least optSearchOption=1.
We need to convince Dogway to implement some of your new features in his very good scripts. ;)

[DTL]

" Intel Compiler. AFAIK it produces the fastest builds, even for AMD."

I tried it in beginning of 202x years and it makes some small visible benefit over VisualStudio compiler. But as we see with Asd-g builds the progress of development of LLVM compilers is good and they makes best binaries. Maybe I also someday will try to install LLVM somehow to my development system.

Current best in quality degrain script with first search with 16x16 blocksize pel=1 and recalculating to 8x8 blocksize pel=2 before MDegrainN (with some not very slow speed):


my_tr=12
my_AMDiffSAD=0
my_thSADA_a=1.2

my_intOvlp=3
my_ovlp=0
my_blksize=16

super_p2=MSuper(last, mt=false, pel=2, hpad=32, vpad=32)
super_p1=MSuper(last, mt=false, pel=1, hpad=32, vpad=32)


vec_am1o4=MAnalyse (super_p1, multi=true, blksize=my_blksize, delta=my_tr, search=3, searchparam=2, truemotion=true, pnew=0, global=true, overlap=my_ovlp, chroma=true,\
optSearchOption=1, optPredictorType=0, mt=false, AreaMode=1, AMoffset=4, AMdiffSAD=my_AMDiffSAD)


multi_vec_mrec_am1o4am2recall=MRecalculate(super_p2, vec_am1o4, thSAD=20, blksize=8, search=3, searchparam=4, truemotion=true, pnew=0, chroma=true, overlap=my_ovlp, \
AreaMode=2, AMoffset=0, tr=my_tr)


MDegrainN(last,super_p2, multi_vec_mrec_am1o4am2recall, my_tr, thSADA_a=my_thSADA_a, thSADA_b=50, mt=false, wpow=4, adjSADzeromv=0.8, adjSADcohmv=0.8,\
thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=my_intOvlp)


The thSAD param for MRecalculate is significant quality/performance balancing value:

1. Low or zero thSAD: Perform refining search of all new recalculated (interpolated) MVs. Slowest mode but may give best quality.
2. Default of 200: Depend on noise level and motion can cause more or less refining searches for blocks with SAD > thSAD.
3. Very high value (about thSCD1 ): Only interpolate MVs and re-check SAD for interpolated MV. Fastest mode but may give lower quality (depend on the noise level/profile and may more).

About low AreaMode setting (starting from 1 'layer'): It looks some better quality happen with non-zero AMoffset value and expected good value about blocksize/4. For higher AreaMode settings the best AMstep/AMoffset values depending on blocksize is still subject of many tests.
For blocksize of 16x16 AMoffset=0 and AreaMode=1 cause only additional search with +-1 of block's center position and it is only 1/16 of block size. With AMoffset=4 more surround samples used in lower number of new search calculations and it looks create more benefit in quality per used CPU cycles.
Current equation for offset from center block's position for each AreaMode step/'layer' (i) is int iOffset = (iAMstep * i + iAMoffset + 1); and new 4 searches of MAnalyse performed with 4 search positions of +-iOffset (diagonal, not sides). So for AreaMode=1, AMoffset=0, AMstep=1 4 new searches performed with +-1 offset (in current 'pel' scale - different for each level and pel setting). It may be not effective (for areas with complex motion) to make additional searches with iOffset > blocksize or even blocksize/2.

Also the AreaMode algorithm may have some 'natural' limit of MVs quality increase with increasing of number of new search positions. Currently internal (not checked) limit of search positions vector is limited to 100 (expected too slow and no one used ever). So user can test AreaMode offsets much larger than blocksize as experiments.

Currently compute complexity is of linear scale from AreaMode setting (number of new search positions is linear scale 4 of AreaMode setting). It is possible to add check of all other possible integer positions around center with given radius (including sides positions and all intermediate). But it will make compute complexity up to square of AreaMode setting. May be in some future version may be added at least sides check option (it only increases complexity 2x linear). Maybe as additional AMflags param like:
AMflags 1 - diagonal offsets
AMflags 2 - sides offsets
AMflags 4 - all offsets in defined by AreaMode+AMstep+AMoffset area.
So user can select AMflags 1+2=3 for diagonal+sides offsets for example. For 'big' blocksizes like 16x16 or 32x32 using AreaMode=1, AMoffset=blocksize/4, AMflags=3 may be better in performance/quality balance. For blocksize 8x8 AMFlags=3 equal to all possible positions with AreaMode=1 (4 diagonal and 4 sides) and so on.

[guest]

I have a question...

In the case of encoding using Distributed Encoding (which can use different CPU's), some are AVX only, some are AVX2, and one has AVX512...

Which "variety" of mvtools would be used (I am yet to be able to test for myself), just curious....

[DTL]

You do not describe how the encoding is distrubuted - by different movies to different hosts or some cut/interleaved parts of single movie to different hosts or even much more methods possible.

For example I now thinking of distrubuting of MAnalyse analysis for AreaMode of different offsets to different hosts if possible and agregate several MVs frames in the single decision filter like MAvg().

For best performance you can install different builds (AVX, AVX2, AVX512) to different hosts. But in some cases the computing results may be somehow different. To get most equal results as possible it may be recommended to install single (lowest SIMD family) build to all hosts and also use SetMaxCPU() to limit all execution to lowest equal SIMD functions.

[guest]

You do not describe how the encoding is distributed - by different movies to different hosts or some cut/interleaved parts of single movie to different hosts or even much more methods possible.

For example I now thinking of distributing of MAnalyse analysis for AreaMode of different offsets to different hosts if possible and aggregate several MVs frames in the single decision filter like MAvg().

For best performance you can install different builds (AVX, AVX2, AVX512) to different hosts. But in some cases the computing results may be somehow different. To get most equal results as possible it may be recommended to install single (lowest SIMD family) build to all hosts and also use SetMaxCPU() to limit all execution to lowest equal SIMD functions.

Hi DTL, glad you asked :)

I use RipBot264 (see sig), and it has this unique feature, that AFAIK, no other encoding app has anything like it.

Staxrip comes close with it's chunk encoding, but it can only use the PC it's being used on :(

RipBot264 can encode "chunks" that the main server PC processes, and then the client (up to 15 other PC's, on a LAN) can then encode these chunks speeding up the process enormously, then when all the chunks are encoded, the client PC's idle, until the main server then combines the chunks, and muxes it all into the completed encoded video/audio file, then if there is another job in the queue, it will commence processing & compiling the chunks to start the process again.

So, to my question, say the server/main PC is AVX2 or AVX512, so you use the AVX2 build of your mvtools, however, some of the PC's in the encoding "farm" are only AVX, or lower, would a script using mvtools (SMDegrain) work on the AVX PC's ??

But for example, if I use the x265 AVX512 command line on my Ryzen 7950X, then the others will automatically disable that command, and proceed.

Would mvtools do something similar ??

Quite interested in the SetMaxCPU(), that might be a good option.

[DTL]

"some of the PC's in the encoding "farm" are only AVX, or lower, would a script using mvtools (SMDegrain) work on the AVX PC's ??"

Script will work if you do not set options with AVX2 or higher requirements (optSearchOption=2 and more). If CPU not supported it must throw error about no AVX2 (or AVX512) present and exit. Also you need to install AVX or lower builds of mvtools2.dll to AVX hosts.

There are 2 different types of SIMD optimizations in current mvtools2 builds:
1. Auto by compiler. These builds can only run at target SIMD architecture (or higher).
2. Manual functions with selectors and architecture check - these are controlled by options (and SetMaxCPU()) and can throw error if options are not compatible with host CPU features present.

Typically performance benefit of auto by compiler optimizations are not big so as safe option you can install SSE2 build to all hosts. And after all scripts testing you can ty to install AVX/AVX2/AVX512 builds to the hosts supported and check the performance difference.

[guest]

Typically performance benefit of auto by compiler optimizations are not big so as safe option you can install SSE2 build to all hosts. And after all scripts testing you can ty to install AVX/AVX2/AVX512 builds to the hosts supported and check the performance difference.

I think the lowest CPU would be AVX, so if I use the AVX compile, on the main server PC, then that should be suitable for any in the "farm" ??

However, if I don't use the AVX CPU's very often, I could use the AVX2 compile...should work either way :)

FYI, I don't need to install it on the clients, as they are "commanded" by the settings on the main encoding server.

I will do some tests, in the coming days, and see what happens.

Thanks for the feedback.

[DTL]

"I think the lowest CPU would be AVX, so if I use the AVX compile, on the main server PC, then that should be suitable for any in the "farm" ??"

Yes.

"I don't need to install it on the clients, as they are "commanded" by the settings on the main encoding server."

It can copy all required .dlls on the clients ? If it can not be disabled - you can only use 'lowest' build on the server and all clients.

[guest]

It can copy all required .dlls on the clients ? If it can not be disabled - you can only use 'lowest' build on the server and all clients.

Not exactly how it does what it does, but it works.

I will test a few different compiles, and let you know how it behaves.

[DTL]

New release: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.29

New params to MRecalculate:

SuperCurrent clip (same as for MAnalyse).

AMthVSMang (float), default = 10.0f (disabled). Threshold of Vectors Stability Metric to skip current level MV and use previous level hierarchy predictor.
Usable range 0.0f .. 1.0f . 0.0f - skip all MVs, 1.0f - do not skip any. Auto-normalized to AMpoints for any combination of AM-params so expected to be non-
dependent on AM-params (AreaMode and others).

AMflags (integer), default = 1. Any combination of AM positions direction flags per each AMstep: 1 = diagonal positions, 2 = sides positions. Current valid
values 1 (4 diagonal positions only), 2 (4 sides positions only), 3 (4 diagonals and 4 sides positions - 8 positions total, about 2x slower).

AMavg (integer), default = 0. The type of averaging operation of MVs after area gathering searches in the checked area around the current block.
0 - Mode (median ?) of dx and dy separated.
1 - Mean of dx and dy.
2 - Mode (median ?) of MVs angle difference.
3 - Mode (median ?) of MVS difference vector length.
Types 0,2,3 uses some performance optimization - skipping of about half search positions if currently gathered MVs equal to input to AreaMode (original block MVs from initial MAnalyse search). Type 1 always searches all defined by AM options positions so may be up to 2 times slower.

New params to MAnalyse:

AMavg (same as for MRecalculate).

AMpt (integer), default = 0. Same as optPredictorType. PredictorType used in AM searches.
AMst (integer), default = 3. Same as search. Search type used in AM searches.
AMsp (integer), default = 2. Same as searchparam. Search param (radius) used in AM searches.


Added some performance optimization for SAD-only DMFlags used (most typical use case).
Fixed optPredictorType=1 for MAnalyse: Added check of median predictor of current level (now zero, global, hierarchy and median predictors checked before refine).

Next ideas for performance/quality balance is to enable use-configurable ratios of already found equal MVs for fast AM-skip processing. Currently if we measure this ratio in the 0.0 to 1.0f range it is already 0.5 for AMavg 0,2,3. But may be extended to much lower than 0.5. It is motion to AM-adaptive search mode. So for stable enough blocks the number of additional AM-searches may be significantly decreased without quality loss. And for non-stable areas the AM searches may be expanded to all defined by settings number of searches (may be several times slower). This level of adaptive search is only possible with per-block AreaMode with onCPU MAnalyse.

In the next versions expected new param like AMfsRatio of the range 0.0f to 1.0f with default of 0.5f (same as current AMavg 0,2,3, optimizations). If set to below 0.5f (or below 1.0f for AMavg=1) it will cause more early (in the AM search positions checking) call to MVs equality comparison function and if all found MVs equal to input (center of block from standard MAnalyse search) it will stop new AM searches and return (output MV assigned as input).

The new added Averaging types for AM may give different results of MVs and resulted in denoising at different types of content (sort of noise profiles) and also personal user preference. In some tests with scanned film footage AMavg=2 saves more details. AMavg=0 makes a bit lower MPEG output bitrate. AMavg=1 provides significantly better denoising (and MPEG output bitrate) but may also cause more details loss.

I made some tests of denoising quality vs Topaz Artemis model: The Topaz AI generally makes much stable objects views in a sequence of frames (even with fast motion and transformed objects) but can lost more details in some frames of low contrast and low brightness areas. mvtools as linear temporal denoiser (also with low enough thSADA_a values around 1..2) left more noise at the fast motion areas and complex transforms but it means these areas left with less processing and close to original input.

[DTL]

Good news on mining cards - prices are really nice like $30..$50 per board like p104-100. And a patched driver promises it can have the NVENC feature working https://github.com/dartraiden/NVIDIA-patcher .

p104-100 board in the best case can be equal to the GTX1070 with 2 NVENC cores working at 1200+fps combined performance for h.264 FHD encoding. So ME performance is expected at the close level. But mining cheap boards really do not have any warranty on NVENC present in working state on chip. So each board requires testing. It is limited to 4x PCIe lines typically but for AreaMode with onboard frames shifting it should not be significantly limiting. The RAM bus of 256bit promises good performance for texture copying for sample-shifted resources onboard.

I am trying to ask p104-100 board sellers to make tests if DX12-ME fully works with patched drivers.

Also as I found from old (2016 ?) years NVIDIA developer papers - the ME-only mode looks like it was working long ago via the NVIDIA SDK (vendor-only API). And it also may accept input MVs as hints for refining. So it is possible to make more quality hierarchy search more close to onCPU MAnalyse by providing the same downsized frames for high levels of search and return interpolated MVs back to ME search engine as hints for lower levels of search.

Some users at github report the MVs input hints feature for ME via DX12 looks like it does not work. It looks like it was not implemented via DX12 drivers from the vendor while announced in the DX12 Microsoft API.

It is from https://forums.developer.nvidia.com/t/question-regarding-10-series-nvenc-encoding/64456

You must accept that NVidia is incapable to describe correctly own products. I lost hundreds of $$$ for NVidia official incorrect or intentionally unpublished information.
https://developer.nvidia.com/video-encode-decode-gpu-support-matrix 38 is wrong again. Line "GeForce GTX 1060 - 1070 Ti" should be "GeForce GTX some 1060 / 1070 / 1070 Ti / 1080" eg. GP104 or should be split to two lines if "GTX some 1060 / 1070 / 1070 Ti" have GP104 but only one hwenc (see next point).
But be warned. Some crippled (with HW faults from chip factory) chips have less CUDA (some faulted SMs are disabled) and sometimes one of two hwenc is faulted too (compare Quadro P4000/P5000 - see https://devtalk.nvidia.com/default/topic/1036615/ 6). So, there may be the same problem with GTX 1070/1070Ti (GP104-200-A1/GP104-300-A1 vs. GP104-400-A1). If https://devtalk.nvidia.com/default/topic/987460/nvdec-cuda-nvenc-speed-comparison/ 16 is correct (eg. tested correctly on real card) GTX 1070 has only one hwenc enabled.
Check also https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units 25 for chip marking and how it is crippled. For example check GTX 1060 - it uses 4 different chips. Normal chip (GP106-400-A1), crippled chips (GP106-300-A1 / GP106-350-K3-A1) and super crippled chip (GP104-140-A1 (only 9 from 20 SMs and 192 of 256 bit memory bus width are working)). ... Life with NVidia is like a box of chocolates, you never know what you're gonna get ! :-)

[takla]

@DTL
https://github.com/ROCm/ROCm/releases/tag/rocm-6.1.0

rocDecode, a new ROCm component that provides high-performance video decode support for
AMD GPUs. With rocDecode, you can decode compressed video streams while keeping the resulting
YUV frames in video memory. With decoded frames in video memory, you can run video
post-processing using ROCm HIP, avoiding unnecessary data copies via the PCIe bus.

Would be nice to have MvTools in HIP

[tormento]

@DTLWould be nice to have MvTools in HIP
If any rewrite occur for GPU, I will prefer CUDA, OpenCL or Vulkan.

CUDA is the fastest, while OpenCL and Vulkan are compatible with almost anything modern.

AMD is prone to present interesting projects, abandon them and never offer proper support.

[DTL]

More real addition to current MAnalyse is optional MVs search via NVIDIA SDK (not using DirectX at all ?). I see the ME-only mode of NVENC noted in the old NVIDIA developer documents around mid-201x. And NVENC SDK/API (encode mode ?) can run from DirectX 9. So it expected to work from Win7 without hacks of DX12 for Win7.

Also NVIDIA SDK in C may be also compatible with UNIX/Linux ? So this mode of MAnalyse will be compatible with both Win and UNIX (as todays no-DX12 builds). But in addition to MVs search only in best case it is required some computing (like SAD shader compute) and also for new AreaMode the onboard resources/texture shift to save CPU time and bus transfers. I expect NVIDIA NVENC SDK resources also somehow can be processed with CUDA or other APIs (without reuploading of resources).

Also it looks NVIDIA do not like to provide all possible features like MVs re-using (as hints) via Microsoft DX API (or too lazy to make all required drivers features for Windows for rarely used API). So making some addition to MAnalyse on NVIDIA SDK/API only have some benefits. But it will be completely NVIDIA-only in hardware. This also may be not great for open source developers.

"AMD is prone to present interesting projects, abandon them and never offer proper support."

Microsoft with Windows looks like still some working global force in the residuals of civilization to force many hardware vendors provide some commonly used hardware features (like general purpose computing and Motion Estimation) via single API. But it looks DirectX only now. And even DirectX (ME) with low number of promised features not completely implemented in some drivers and/or for some products.

[takla]

If any rewrite occur for GPU, I will prefer CUDA, OpenCL or Vulkan.


OpenCL is superseded by Vulkan.

Also
https://i.imgur.com/n0XFBoN.jpeg

[tormento]

More real addition to current MAnalyse is optional MVs search via NVIDIA SDK
Are you talking about this (https://developer.nvidia.com/optical-flow-sdk)?

[DTL]

About this: https://developer.nvidia.com/video-codec-sdk

Currently implemented in mvtools2:
DirectX Video

Platform Windows

Benefits
Low Level Control
Native DirectX and Windows Integration
Easy for DirectX developers
Multi-Vendor

Native API interface - D3D11 (Decode only) and D3D12

So it is Windows only and Win10 with DX12 and higher only. But support any (all 2 current) manufacturers of DX12 boards - NVIDIA and AMD and possibly some or some days of intel DX12).

Possible other API:

NVIDIA Video Codec SDK
Platform Windows and Linux

Benefits
High Level Control
Native Integration in custom pipelines
Useful for users with less knowledge of Vulkan and Direct X
Easy for C, C++ developers
Nvidia Proprietary API
Comprehensive feature set

Native API interface
D3D9, D3D10, D3D11, D3D12 (Encode only) CUDA (Encode and decode)

So it is Linux and Windows compatible (more happy opensource developers from the residuals of current White civilization) and Windows 7 compatible with DX9 (more happy users of Win7). Also possible more features available. But main disadvantage: NVIDIA hardware only.

Made some testbuild for possible peak ME performance test for multi-positions search for AreaMode: https://drive.google.com/file/d/1gJsSWLpSrtUtEfJkjtAlaC0sauYkcirL/view?usp=sharing

It have 3 .dlls :
x1me - standard 1 call to motion estimation engine per input pairs of frames
x5me - 5 sequential calls to motion estimation engine per input pairs of frames
x9me - 9 sequential calls to motion estimation engine per input pairs of frames

AreaMode=1 (AMflags=1) will use 5 searches for single pairs of frames and AreaMode=2 (AMflags=1) will use 9 searches.

Test script:

LoadPlugin("mvtools2_x9me.dll")

ColorBars(1920,1080, pixel_type="YV12")
#ColorBars(3840,2160, pixel_type="YV12")

#super = MSuper(mt=false, pel=4, pelrefine=true, chroma=true, levels=0) # onCPU compare
super = MSuper(mt=false, pel=4, pelrefine=false, chroma=true, levels=1) # onHWA
#forward_vec1 = MAnalyse(super, isb = false, delta = 1, search=3, chroma=true, optSearchOption=1, levels=0) #onCPU cpmpare
forward_vec1 = MAnalyse(super, blksize=8, isb = false, delta = 1, chroma=true, optSearchOption=5, levels=1) # onHWA
MStoreVect(forward_vec1)

Prefetch(...)


With Gigabyte GTX1060 card GPU clock max 1911 MHz and 1920x1080 frame:

x1me - 500 fps, 74% load of Video Encoder. Performance somehow limited to data transfer overheads. Bus Load ~30%, GPU chip power ~35W only. Expected ME RAW frames pairs per second performance 500/0.74 = 675 fpps.

x5me - 150 fps, 100% load Video Encoder. Bus load ~8%. ME RAW frames pairs per second performance 150x5 = 750 fpps.

x9me - 84 fps, 100% load Video Encoder. Bus load ~5%. ME RAW frames pairs per second performance 84x9 = 765 fpps.

So single NVENC at Pascal chip as expected provides ME performance about 750 fpps. Full-blood GTX1070 and 1080 boards with Dual NVENC healthy expected about 1500 fpps ME RAW performance.

The i5-9600K provides about 6 times lower (with same pel=4 quality).

Changing blocksize from 8x8 to 16x16 do not change anything - it looks performance of NVENC is only samples limited and not blocks number (per frame or per second). With 3840x2160 frame it is about 4x linearly lower.

Expected performance of AreaMode=1 AMflags=1 for 1920x1080 frame size and tr=12:
tr=12 need 24 frames pairs per output frame and AreaMode=1 AMflags=1 need 4 additional frames pairs search (x5 total) so MDegrainN output frame require 24*5=120 pairs frame search with these settings. 750 fpps ME RAW performance of single Pascal NVENC divided to 120 ~= 6 fps total denoise performance max.

With GTX1070..1080 boards (p104..p102 miners editions too ?) performance expected twice higher.

Table 3 of the document https://developer.download.nvidia.com/designworks/video-codec-sdk/secure/7.1/01/NVENC_DA-06209-001_v08.pdf also lists about 648 fps for Pascal NVENC at h.264 encoding at fast mode with FHD frame.

Another perspective Average function to AMavg planned is geometric median - https://en.wikipedia.org/wiki/Geometric_median . It is not easy to compute so may add more performance penalty for onCPU MAnalyse. But planned to offloading to Compute Shader for onHWA processing in the future.
Valuable property of geometric (2D) median for noised sources is:
The geometric median has a breakdown point of 0.5.[13] That is, up to half of the sample data may be arbitrarily corrupted, and the median of the samples will still provide a robust estimator for the location of the uncorrupted data.

[tormento]

About this
Have a look at the link I provided. It seems to have everything ready.
Full-blood GTX1070 and 1080 boards with Dual NVENC healthy expected about 1500 fpps ME RAW performance.
I suppose you are aware of NVEnc unlocking patches.
p104..p102 miners editions too
There are patches for them too. :)

[DTL]

"It seems to have everything ready."

That promo do not list any denoising application. So it may work good only with new cameras at good light with low noise at input. Same as RIFE also fail with noised sources. Also I do not have Turing board to check it.

"I suppose you are aware of NVEnc unlocking patches."

I do not got any out_of_licence or other error. It simply limits by expected max MPEG encoder rate at lowest settings (fastest). So I think it is natural limits of NVENC hardware. Also licence limits MPEG encoding sessions to 2 for end-user drivers. Here we have usage of 1 D3D12 device by 1 process. And even not in MPEG encoding mode. May be it is too rarely used by anyone at this civilization so NVIDIA do not think about putting any limits to this API. It even do not use any MPEG-LA licensed algorithms I think.

[guest]

Originally Posted by DTL

It can copy all required .dlls on the clients ? If it can not be disabled - you can only use 'lowest' build on the server and all clients.

Not exactly how it does what it does, but it works.

I will test a few different compiles, and let you know how it behaves.

Hi DTL, well, I've finally got around to doing some test's, and most of your current builds work for me, (on the AVX2 CPU's) but as I tried to explain the Distributed Encoding I use, the clients that only have AVX CPU's, do not start encoding, despite what compile I tried, when using an SMDegrain script that calls mvtool2.dll.

Not sure what to do, now :(

[DTL]

More safely use _e.XX builds for SMDegrain. Latest is https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-e.03 . If it also not work on AVX - the only way is fallback to pinterf' 2.7.45 builds.

[guest]

More safely use _e.XX builds for SMDegrain. Latest is https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-e.03 . If it also not work on AVX - the only way is fallback to pinterf' 2.7.45 builds.

Hi DTL,

Another day, another lot of tests to figure out what's going on here...

So I thought I'd do it all on the AVX E5-2697 v2 dual CPU system, cut to the source, (so to speak), and I've uncovered several problems :(

I was using AVX2 builds of HDRTools & plugins_jpsdr that were causing unexpected problems, so I have revised that situation, and then I tried your latest compile of mvtools,
and I used the AVX DX12 (I have DX installed on this particular PC), and had no issues, tried noDX12, that worked. (any reason why you've done DX variants ?)

Have not got an AVX PC without DX installed, so I can't check that.

So after all that fiddle, I think I've got it sorted, next test is that it behaves itself using the Distributed Encode function from another AVX2 PC (the Ryzen's), in which I will use the DX12 AVX compile.

[DTL]

"any reason why you've done DX variants ?"

They are for hosts where DX12 installed and users want to use DX12 features. If DX12 is not installed - they will not loads at all. So for Win7 and old only noDX12 builds are applicable.

[guest]

"any reason why you've done DX variants ?"

They are for hosts where DX12 installed and users want to use DX12 features. If DX12 is not installed - they will not loads at all. So for Win7 and old only noDX12 builds are applicable.

All my PC's are running Windows 11.

So noDX12 ?

I do have DX installed, and both variants worked.

[DTL]

Win10 and later looks like has DX12 embedded from setup - you can use both DX12 and noDX12 builds.

If you have different CPUs in the farm - can you test onCPU MAnalyse performance using this script:


LoadPlugin("mvtools2.dll")

ColorBars(1920,1080, pixel_type="YV12")

super = MSuper(mt=false, pel=4, chroma=true)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, search=3, chroma=true, mt=false)
MStoreVect(forward_vec1)

Prefetch(...)


with AVSmeter ? I hope this can run with any build of mvtools. Its output is RAW ME performance in frames pairs per second (using Expanding search - close to Exhaustive).

Expanding (from zero dx, dy) center search with pnew > 0 (better to adjust using MShow or other methods for current noise in the source) expected to be more noise-resistant at the static areas. Because after checking center (not moving point) - the other definitely false positions will be somehow penalty-protected by pnew setting. Though too high pnew param will cause skipping of real motion and blur in the motion areas.

Where Prefetch() is for number of real CPU cores at host.

[guest]

Win10 and later looks like has DX12 embedded from setup - you can use both DX12 and noDX12 builds.

If you have different CPUs in the farm - can you test onCPU MAnalyse performance using this script:


LoadPlugin("mvtools2.dll")

ColorBars(1920,1080, pixel_type="YV12")

super = MSuper(mt=false, pel=4, chroma=true)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, search=3, chroma=true, mt=false)
MStoreVect(forward_vec1)

Prefetch(...)


with AVSmeter ? I hope this can run with any build of mvtools. Its output is RAW ME performance in frames pairs per second (using Expanding search - close to Exhaustive). Expanding (from zero dx, dy) center search with pnew > 0 (better to adjust using MShow or other methods for current noise in the source) expected to be more noise-resistant at the static areas. Because after checking center (not moving point) - the other definitely false positions will be somehow penalty-protected by pnew setting. Though too high pnew param will cause skipping of real motion and blur in the motion areas.

Where Prefetch() is for number of real CPU cores at host.

I have to say that I have no idea how to "run" that script with AVSMeter :(

I am only familiar with how scripts are written for RipBot, so changing that script for RipBot is way over my head, sorry.

Here is a sample of a script, from RipBot:-

#After_Prefetch_Custom
LoadPlugin("%AVISYNTHPLUGINS%\Plugins_JPSDR\Plugins_JPSDR.dll")
LevelLimit=(video.BitsPerComponent==8) ? 255 : 1023
IntensityMask=ConvertToY(video).Levels(0,2,LevelLimit,0,LevelLimit,coring=false)
EdgeMask=aSobel(IntensityMask,chroma=0,thresh=255,SetAffinity=false).invert.Levels(0,2,LevelLimit,0,LevelLimit,coring=false).Blur(1)
SharpMask=Overlay(IntensityMask,EdgeMask,mode="Multiply",opacity=1.0)
SharpenedVideo=Sharpen(video,1)
video=Overlay(video,SharpenedVideo,mask=SharpMask,opacity=1.0)

[DTL]

"I have to say that I have no idea how to "run" that script with AVSMeter"

1. Enter required number of threads for Prefetch(arg) and save to .avs file like test.avs (you can use Notepad text editor from Windows).

For 2 cores CPU (Core2Duo E7500)

LoadPlugin("mvtools2.dll")

ColorBars(1920,1080, pixel_type="YV12")

super = MSuper(mt=false, pel=4, chroma=true)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, search=3, chroma=true, mt=false)
MStoreVect(forward_vec1)

Prefetch(2)


2. Put all required files to some folder like c:\test :
c:\test\test.avs
c:\test\mvtools2.dll
c:\test\AVSmeter64.exe

3. Run from command prompt

c:\test\AVSmeter64.exe test.avs

It will shows performance metering like

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r3982, 3.7, x86_64) (3.7.3.0)

Number of frames: 107892
Length (hh:mm:ss.ms): 00:59:59.996
Frame width: 720
Frame height: 180
Framerate: 29.970 (30000/1001)
Colorspace: RGB32

Frame (current | last): 2416 | 107891
FPS (cur | min | max | avg): 6.130 | 3.900 | 317501 | 19.94
Process memory usage: 346 MiB
Thread count: 8
CPU usage (current | average): 100.0% | 93.9%


Better to wait some time for FPS avg value to become stable.

Where required line only
FPS (cur | min | max | avg): 6.130 | 3.900 | 317501 | 19.94

If Prefetch( ) arg set correctly (not very low for current CPU cores present) - AVSmeter must display CPU usage close to 100%.

By placing different mvtools2.dll builds in the folder with script you can also check if any performance difference exist between SSE2/AVX/AVX2 builds at your host. If you need best performance with most common scripts (QTGMC/SMDegrain) without modification - I think -e.XX builds will be faster. For -a.XX builds it is recommended at least add optSearchOption=1 to all MAnalyse calls to enable some SIMD optimizations.

[guest]

"I have to say that I have no idea how to "run" that script with AVSMeter"

1. Enter required number of threads for Prefetch(arg) and save to .avs file like test.avs (you can use Notepad text editor from Windows).

For 2 cores CPU (Core2Duo E7500)

LoadPlugin("mvtools2.dll")

ColorBars(1920,1080, pixel_type="YV12")

super = MSuper(mt=false, pel=4, chroma=true)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, search=3, chroma=true, mt=false)
MStoreVect(forward_vec1)

Prefetch(2)


2. Put all required files to some folder like c:\test :
c:\test\test.avs
c:\test\mvtools2.dll
c:\test\AVSmeter64.exe

3. Run from command prompt

c:\test\AVSmeter64.exe test.avs

It will shows performance metering like

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r3982, 3.7, x86_64) (3.7.3.0)

Number of frames: 107892
Length (hh:mm:ss.ms): 00:59:59.996
Frame width: 720
Frame height: 180
Framerate: 29.970 (30000/1001)
Colorspace: RGB32

Frame (current | last): 2416 | 107891
FPS (cur | min | max | avg): 6.130 | 3.900 | 317501 | 19.94
Process memory usage: 346 MiB
Thread count: 8
CPU usage (current | average): 100.0% | 93.9%


Better to wait some time for FPS avg value to become stable.

Where required line only
FPS (cur | min | max | avg): 6.130 | 3.900 | 317501 | 19.94

If Prefetch( ) arg set correctly (not very low for current CPU cores present) - AVSmeter must display CPU usage close to 100%.

By placing different mvtools2.dll builds in the folder with script you can also check if any performance difference exist between SSE2/AVX/AVX2 builds at your host. If you need best performance with most common scripts (QTGMC/SMDegrain) without modification - I think -e.XX builds will be faster. For -a.XX builds it is recommended at least add optSearchOption=1 to all MAnalyse calls to enable some SIMD optimizations.

OK, thanks for that, I will attempt to do this in the next few days.

Most of my CPU's are 12 or 16 core Ryzens.

The AVX CPU's are Xeon E5 2697 v2 (12c dual)

[guest]

2. Put all required files to some folder like c:\test :
c:\test\test.avs
c:\test\mvtools2.dll
c:\test\AVSmeter64.exe

3. Run from command prompt

c:\test\AVSmeter64.exe test.avs

It will shows performance metering like

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r3982, 3.7, x86_64) (3.7.3.0)

Number of frames: 107892
Length (hh:mm:ss.ms): 00:59:59.996
Frame width: 720
Frame height: 180
Framerate: 29.970 (30000/1001)
Colorspace: RGB32

Frame (current | last): 2416 | 107891
FPS (cur | min | max | avg): 6.130 | 3.900 | 317501 | 19.94
Process memory usage: 346 MiB
Thread count: 8
CPU usage (current | average): 100.0% | 93.9%


Better to wait some time for FPS avg value to become stable.

Where required line only
FPS (cur | min | max | avg): 6.130 | 3.900 | 317501 | 19.94

So I had the opportunity to do some test today, and they are interesting, almost like a benchmark test for Prefetch.

One thing that was strange, I did many test's with different mvtools.dll, and Prefetch's, and then just to double check,
I did a couple that had good results on the 1st run, but on the 2nd run were quite different, faster.....:confused::confused::confused:

Anyway, these were all done on an AVX CPU system (E5-2697v2)

Is there any reason to test on an AVX2 PC ??

Anyway, here are the results:-

https://www.mediafire.com/file/1igo02cyhuq2j27/results.txt/file

I did some quick tests on the Ryzen 7950X

https://www.mediafire.com/file/m6q07md79n8p4ki/7950X.txt/file

[guest]

LoadPlugin("mvtools2.dll")

ColorBars(1920,1080, pixel_type="YV12")

super = MSuper(mt=false, pel=4, chroma=true)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, search=3, chroma=true, mt=false)
MStoreVect(forward_vec1)

Prefetch(2)



This is probably a bit off topic, but would there some way to use a similar "test" to benchmark x265 @ 4K, using different Prefetch settings ??

[DTL]

x265 is completely separate process from AVS+ system process so it can not be controlled by AVS+ threading settings. x265 uses its own methods of multithreading and provides separate controls (limited ?).

[guest]

x265 is completely separate process from AVS+ system process so it can not be controlled by AVS+ threading settings. x265 uses its own methods of multithreading and provides separate controls (limited ?).

So no comments on the mvtools tests ????

[DTL]

I am missed that previous post. Will look now or later when have time.

Quick answer:

"Is there any reason to test on an AVX2 PC ??"

Yes - the main steps in SIMD designs are SSE2 (or latest 4.2) and next is AVX2 for full-blood integers processing. AVX is only new beginning of 256bit SIMD and mostly limited to rarely used float calculations. Next step to 512bit is AVX512.

So I typically make builds of SSE2/AVX2/(AVX512).

[DTL]

Finally some time to type the long post:

Thank you for the provided test results. One valuable result is that even top end-users desktop CPUs are still significantly slower in ME in comparison with 10years old MPEG encoder chips. Also it takes close to 100% CPU time and nothing left on MDegrainN and MPEG encoding (x264/x265).

First the recommended number of Prefetch threads is the number of physical cores, not total logic if HyperThreading is enabled. Though it may greatly depends on each CPU architecture especially in the case of massive-multicores chips with low RAM channels and big enough cache.

I also made some tests with that script and CPUs and NVIDIA chips.

It looks the performance is heavily limited by RAM performance so at some CPUs the total cores and 50% cores performance is very close. It means for a real transcoding process it may be better to finetune threads number on the AVS script used so it can leave more CPU time for MPEG encoder. In complex scripts with mvtools used it may be tested with a different Prefetch() setting for each filter (or groups of filters ?)

Like

_myDegrain_with_mvtools_script(params).Prefetch(K)

other_plugins_calls(params).Prefetch(N)

where K < N.

And adjust N and K numbers for best total transcoding performance at a given host.

i5-9600K (6 cores without HT) example:
Prefetch(6) - 120 fps (97% total CPU load)
Prefetch(8) - 118 fps
Prefetch(4) - 120 fps (66% total CPU load)
Prefetch(3) - 112 fps (50% total CPU load)

The on chip cache size looks like winning the game with AMD 7950X (64 MB cache) even in comparison with Xeon Gold 6134 (8c/16th 32 MB cache) and 6 (?) RAM channels.
Xeon Gold 6134 results vs Prefetch number
Prefetch(8) - 220 fps (49% CPU load)
Prefetch(12) - 248 fps (74% CPU load)
Prefetch(16) - 258 fps (91% CPU load)

While with a fixed AVSTP plugin you can test internal MT in mvtools and it may make more performance benefit because of lower RAM usage and more on-chip cache efficiency. Unfortunately -a.XX builds long ago were not tested with AVSTP and look like they become very unstable in that mode.

Testing of different builds branches (-a.XX and -e.XX) really not completely fair as I remember a.XX have always enabled some simple logical optimization of skipping of already checked predictors and it can make visible performance benefit with static noise-free sources like ColorBars(). An attempt to add noise with AddGrain() will take some CPU time and lower results. 2.7.45 build and -e.03 build do not skip any predictor check and make some redundant work and it causes more or less visible performance penalty (may significantly depend on source).

As I see the AMD 7950X also have some graphic core integrated - you can also test its performance in hardware ME mode (using DX12 builds of a.XX and put Compute.cso with same folder) if it provide this API:

LoadPlugin("mvtools2.dll")

ColorBars(1920,1080, pixel_type="YV12")

super = MSuper(mt=false, pel=4, chroma=true, levels=1, pelrefine=false)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, chroma=true, mt=false, optSearchOption=5, levels=1)
MStoreVect(forward_vec1)

Prefetch(..)


Also found during DX12 tests: Setting MT mode to 3 greatly limits performance of DX12, so default MT mode does not limit performance.

Tested Maxwell 2nd gen - it does not provide DX12ME API. So it looks like everything usable starts only from the Pascal series.

Turing cards with single NVENC are only slightly faster over Pascal 1 NVENC (about 700 vs 600 fpps).

I tried to find any information on possibly applicable AMD cards - found only some Wiki about different generations of MPEG hardware encoding ASIC from 199x to 202x on ATI/AMD cards - it looks like only modern VCE can be usable ?

Next release will have MAverage onCPU script function for performing script-based AreaMode simulation. First tests show with pel=4 precision it runs about 4 times faster (i5-9600K and GTX1060) in comparison with onCPU at AreaMode=1 AMflags=1 search more (+4 additional search positions for block). And still uses only 45% of single NVENC. So with full DX12 onboard frame shifts it is expected to be about 2 times faster. But DX12 onboard frame shifts requires much more DX12 programming. The quality is still not as perfect as expected - need more debugging. Currently SAD (DM) is not updated in simple MAverage function (DM update requires much more programming with import of super clip too). But if using IntOvlp=1 or 3 the SAD must be updated in MDegrainN after MVs interpolation so precheck of SAD in MAverage may be redundant and cause performance penalty. Need to check what is wrong now with current sources (some bad blocks passed to output like SAD not updated for resulting MVs or maybe other bugs still exist).

[guest]

Finally some time to type the long post:

Hi DTL,

You are quite good at long posts :)

So most of that doesn't mean too much to me, but if you got some of that from my tests, then it was worth it.

I might do the same tests on my 13900KF (no iGPU), and see what that shows with the P & E cores.

The 7950X does have an iGPU, but it's so "weak" I don't enable it, or install the appropriate drivers.

Regards

[DTL]

"The 7950X does have an iGPU, but it's so "weak" I don't enable it, or install the appropriate drivers."

In the reviews it is named as RDNA2 and can encode 264/265 and DX12_1 (or even DX12_2) compatible - so you can try enable it and check performance if it provides DX12-ME API too.

[guest]

"The 7950X does have an iGPU, but it's so "weak" I don't enable it, or install the appropriate drivers."

In the reviews it is named as RDNA2 and can encode 264/265 and DX12_1 (or even DX12_2) compatible - so you can try enable it and check performance if it provides DX12-ME API too.

There's probably not much point in testing it, as I need to compensate for the lesser CPU's used in the encoding pool.

[takla]

@DTL
I've been away for a while.
Have you managed to implement adaptive-thSAD ?
I mean thSAD automatically calculating and applying the optimal value on a per-frame basis?

And have you managed to add AVX 512 support?

[DTL]

It was somehow simply implemented in a.XX and also e.XX builds - https://forum.doom9.org/showthread.php?p=1990642#post1990642 . But adjustment of scale and offset params may significantly depends on both source 'noise type/level' and also user's expectation on thSAD used. So thSADA_a may be tweaked from about 1.0 for medium (or close to minimal already started) noise reduction to 2..3+ for much higher (also may cause details blur as usual with too high thSAD).

About future development - as I read developers of MPEG encoders finally understand benefit of more or less noise reduction and start to integrate spatio-temporal noise reduction in MPEG encoder engines too (in new x265 builds ?). So they can reuse MVs for both noise reduction and MPEG encoding and it is faster. It was planned in old years to feed best and very expensive found MVs from degrainer ME into MPEG encoder engine to save some time and increase quality on MPEG compression. But it looks civilization degrade at too large speed and about zero developers for denoisers left. Last developers of this civilization are around MPEG encoders like x264 and x265 (and possibly some later somehow a bit advanced) and they finally start to do good progress in understanding of natural moving pictures compression. If they like they can reuse all currently implemented (and planned and described) ME features of mvtools project. Though higher quality features like Area Mode search are also very compute extensive and slow.

I do not have AVX512 development and test setup also and our company do not plan to make hardware upgrade in any close years. May be to the end of 202x or somewhere in 203x we will finally got some thrown-away old or very cheap new like $30 chips with AVX512 to continue development. AVX512 can somehow help to get more data at AreaMode several shifted positions search at the same time. But its programming require lots of brain resources and if about no one uses these slow modes so very few reasons to do it. In next years we can test and look for the progress of internal spatial-temporal noise reduction engines in the MPEG encoders and compare with mvtools-based pre-filtering. May be better will be some workload separation between some prefiltering with mvtools and next stage internal temporal noise reduction in MPEG encoder. Also in next years it is possible to see hardware temporal noise reduction in new MPEG encode ASICs and it may be much faster in comparison with mvtools onCPU.

[takla]

It was somehow simply implemented in a.XX and also e.XX builds - https://forum.doom9.org/showthread.php?p=1990642#post1990642 . But adjustment of scale and offset params may significantly depends on both source 'noise type/level' and also user's expectation on thSAD used. So thSADA_a may be tweaked from about 1.0 for medium (or close to minimal already started) noise reduction to 2..3+ for much higher (also may cause details blur as usual with too high thSAD).

Too bad. I was hoping a simple internal function in your mvtools2 fork, like lets say adaptive-thSAD=true/false
and then maybe two limiters for min-thSAD & max-thSAD.


About future development - as I read developers of MPEG encoders finally understand benefit of more or less noise reduction and start to integrate spatio-temporal noise reduction in MPEG encoder engines too (in new x265 builds ?). So they can reuse MVs for both noise reduction and MPEG encoding and it is faster. It was planned in old years to feed best and very expensive found MVs from degrainer ME into MPEG encoder engine to save some time and increase quality on MPEG compression. But it looks civilization degrade at too large speed and about zero developers for denoisers left. Last developers of this civilization are around MPEG encoders like x264 and x265 (and possibly some later somehow a bit advanced) and they finally start to do good progress in understanding of natural moving pictures compression. If they like they can reuse all currently implemented (and planned and described) ME features of mvtools project. Though higher quality features like Area Mode search are also very compute extensive and slow.

It really is like that. Especially the "understanding of natural moving pictures compression"
Sometimes it feels like a global consciousness, where X amount of people have to spend Y amount of time to "unlock" that knowledge for more people. Sure you can say "thats just what research papers are for" but I feel it is higher than that. Its not even about the information itself but the understanding of information processing.


I do not have AVX512 development and test setup also and our company do not plan to make hardware upgrade in any close years. May be to the end of 202x or somewhere in 203x we will finally got some thrown-away old or very cheap new like $30 chips with AVX512 to continue development. AVX512 can somehow help to get more data at AreaMode several shifted positions search at the same time. But its programming require lots of brain resources and if about no one uses these slow modes so very few reasons to do it. In next years we can test and look for the progress of internal spatial-temporal noise reduction engines in the MPEG encoders and compare with mvtools-based pre-filtering. May be better will be some workload separation between some prefiltering with mvtools and next stage internal temporal noise reduction in MPEG encoder. Also in next years it is possible to see hardware temporal noise reduction in new MPEG encode ASICs and it may be much faster in comparison with mvtools onCPU.

Yeah. Something like saving motion-vectors to a file should have been a thing from the very start.
Just thinking about it, you could have someone do the slowest possible mv settings, and then share that file for people who own the exact same video stream, e.g. Blu-Ray. It would save a lot of energy and time.

Also, thank you for bringing it to my attention that x265 developement has continued (https://x265.readthedocs.io/en/master/releasenotes.html#version-3-6)

[DTL]

" like lets say adaptive-thSAD=true/false"

You can simply set params to some fixed values like thSADA_a=1.2 and thSADA_b=50. But to have control for thSAD decreasing with more tr from current frame - user still need to control thSAD and thSAD2 values (or left defaults). Also as used thSAD anyway significantly change the 'denoising power' it is required some control over its 'power'. It is no good to set 'full-auto thSAD' - it will cause too low denoising for one user or too high to other and there will be no way to control over it. Now we have most significant control param of thSADA_a of values like:
1.0 - low
2.0 - medium
3.0 - high

With float (close to unlimited) precision for fine-tuning. And also second additive param thSADA_b to shift all values a bit higher or lower unconditionally (you can in theory set thSADA_a=0 and thSADA_b=const to completely disable per-frame thSAD variability and it is equal to const thSAD). So thSADA_a mostly control per-frame variability of thSAD used and also possible to create different versions of auto and fixed.

"maybe two limiters for min-thSAD & max-thSAD."

Not yet implemented. It is simple and good idea but require more params like thSADA_min and thSADA_max. Easy to do. Add limiting to min and max to calculated value at https://github.com/DTL2020/mvtools/blob/9eedb9d0850f638fc43212fb515cf048f9b9a58f/Sources/MDegrainN.cpp#L7440

"Just thinking about it, you could have someone do the slowest possible mv settings, and then share that file for people who own the exact same video stream, e.g. Blu-Ray. It would save a lot of energy and time."

Yes - process of searching for best possible MVs is very compute loaded so can be done as distributed computing (the workunit can be some frames range + tr frames around edges of range). Also it is only translate transtform MVs - no other possible transforms like rotation-lighting-scaling and many others currently in searching. But for best possible MVs it is good to make very many passes analysis over all movie (all typical 130000 frames of 90 min 24fps movie). Because it is iterative process of search and refining (by denoising) of objects patches. So if current scene only have too bad patch view severily distorted by noise (like low-light scene with high ISO/gain) but other scene can have this patch texture much better quality so best search engine need to look over all possible frames. It is very complex and compute-loaded task so very slow. Also currently there is no real 'exhaustive' search - only highly optimized multi-step multi-resolution and it can miss some best MVs too. Full ESA search is sort of radius=frame_size/2 and not simply several samples around current. And it is several decimal orders of magnitude slower too. Also it can be multiplied to complexity of AreaMode like search close to each sample + its surroundings individual search instead of much smaller blocks grid search.

The resulted MVs files are small enough and can be released for each known 'classical/fixed' title and can be refined by many users in parts or complete files as different versions from each user. Though there are many possible 'overlayed' blocks scans (subdivision of frame into blocks) in the mvtools - each overlay combination (block size + V and H overlap) create different MVs file.


" x265 developement has continued"

New feature
Motion-Compensated Spatio-Temporal Filtering.

Yes - it is finally beginning of the complex in design and very compute-complex part of MPEG encoder to skip more random data typically from medium distortions (film grain and photon shot noise and electronic other noises) and to keep and save more real object's data. Its advancement may lasts to the last programmers of current civilization. Though first implementations in MPEG encoder may be as simple as initial mvtools design from 2004.

I hope new internal temporal denoisers will be added someday to x264 encoder too.