New DirectX 12 Video APIs

[DTL]

"Is HBD a long way to go?"

It not supported by DX12_ME for ME work. Only NV12 is the only supported input format currently for ME engine. It is for bitdepth equal to YV12 8bit format (difference is only in interleaving UV planes). So is the current requirement for SO=5 mode of MAnalyse - only YV12 input. It is currently converted to NV12 at the upload procedure - Y plane used directly as first subresource slice and interleaved UV as second subresource slice. In theory it is possible to upload more bits formats and convert to NV12 by compute shader but it is not first place work. Better use ConvertBits(8) or ConvertToYV12() for the MAnalyse source. Though using 8bit only may limit quality of processing HDR sources (at dark areas) - so it may be recommended to convert HDR to SDR 8bit before feeding MAnalyse.

To have some 'more accelerated' HBD - either required to design compute-shader based MAnalyse (SearchMVs() function of PlaneofBlocks) or MDegrain. It is all planned but in some future. I hope compute-shader based MAnalyse will be faster in compare with the dedicated ME engine from encoder. Currently 'GPU load' system stats shows almost 0 load (even with using SAD compute shader). So it is still free compute resources exist.

The only way to increase speed of ME engine of encoder is possibly overclock accelerator (or measure its speed in different chips).
May be chips with more 'encoder units' will have also ability to run more ME threads at the same time. Currently DX12 environment looks like completely not limits number of ME threads (for same process or different processes) so it is hard to find what is the actual limit of performance of current hardware. One possible method is to run MAnalyse + MDegrain with AVSmeter and get close to 100% Video Encode load and check for output fps and tr-value. The performance of ME engine is about fps*(2*tr) pairs of frames per second.

The MDegrainN support HBD input/output already for onCPU processing. So it looks like required to create 2 'super' clips for using in MAnalyse (8bit) and in MDegrain (>8bit).

The addition of compute-shader based MDegrain is in progress. It started from uploading all required resources (2 x tr + 1 number of frames) in HWAcc. It also will make speed of MAnalyse best possible (only 1 new upload frame per 1 output frame of MDegrain). Currently MAnalyse still use partial optimization - 1 src frame is remembered and not uploaded for all src+ref pairs requested by MDegrain for each output frame. So with full optimization the upload traffic will be (2 x tr) times smaller (also leave more host memory speed for other tasks). Currently even making Y-plane not copy in NV12 intermediate structure before upload makes visible speedup.

The expected limiting factor for MT Avisynth - too low onboard memory of accelerator to hold total frames pool of all MAnalyse+MDegrain threads. With current architecture the frames pool can not be shared across different AVS threads. So it possibly will not allow to run script with all possible CPU cores threads number (but it will leave more free cores to MPEG encoder). With shared ME engine for all threads it may be no need to run many threads to get max possible speed. So the Prefetch(N) should be adjusted between accelerator memory load and Video Encode load. If Video Encode is close to 100% load - no need to add more threads.

Current major limitation of usage of DX12_ME engine - it looks overlap processing mode (for MAnalyse) is not possible (keeping speed in good range). Some cosplay of overlap possible only by sending 4 shifted planes to process but it will make speed 4 times slower. So the only way to get less blocky output of MDegrain is new no-overlap blending mode (still work in progress) with smooth transition of blending weight between blocks. It may run more or less slower onCPU but should run good at compute shader based processing. Though with 'half-frame' SO=6 mode the frame size is 4 times smaller and possibly it will be processed 4 times faster by ME engine and allow to run some cosplay of overlap mode with good speed - need testing too. And SO=6 mode currently limits usable block size to 16x16 only.

[magnetite]

I kept getting this unhandled C++ exception error while trying to run the new version. However, running it with MeGUI's manual mode actually gave me a specific error message, instead of some generic unhandled C++ exception:

Avisynth script error:
Unhandled error: dimzon_avs_init_2


LoadPlugin("C:\MeGUI 64-bit\tools\lsmash\LSMASHSource.dll")
LSMASHVideoSource("E:\Mobile Video\MVI_0564.mp4")
ConvertToYV12()
tr = 2 # Temporal radius
super = MSuper (mt=true, chroma=true,pel=1, hpad=8, vpad=8)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, chroma=true, overlap=0, mt=true, optSearchOption=5, levels=1)
MDegrainN (super, multi_vec, tr, thSAD=300, thSAD2=290, mt=true, wpow=4)
Prefetch(8)


Removing the optSearchOption=5 and levels=1 fixes it.

[DTL]

Is it keep to throw error in 1 thread mode ? Without or commented-out Prefetch(8) at the end of script.

I had some Unhandled C++ exception inside Avisynth when trying to use some Graphic lib from DX12TK in multithreaded mode but now it do not use it.

Also try to set mt=false for MSuper (and also for MAnalyse !)- may it somehow conflict with something (if you also have avstp.dll somewhere and avstp-mt is really activated). With SO=5 the processing in MAnalyse should not reach the avstp-mt slicing now but it still may somehow made difference.

As for MDegrainN avstp-mt - not sure how it work - I not use it and not tested. First it is also recommended to set mt=false in all mvtools filters.
Also if you use AVS+ mt with Prefetch(N) at the end - it mostly good to disable any avstp-mt in mvtools because it will makes things much more slow. You need to use either avstp-mt or AVS+ mt or precisely adjust threading of avstp-mt with AVS+ mt if avstp-mt is really faster in your case.

If avstp-mt is activated in your environment with that script it try to create 8+(num of cores ?) threads. avstp-mt in mvtools is now 'shadow of the poor past' and only may be recommended to use with care if AVS+ mt is not possible.

Also you may set levels=1 to
super = MSuper(chroma=true,pel=1, hpad=8, vpad=8, levels=1)

MAnalyse with SO=5 do not use >1 level now so it may make processing a bit faster (no need to generate lower res levels in MSuper). MDegrainN also do not use smaller levels.

[magnetite]

I don't use avstp. Just use Avisynth+'s MT modes. With MT completely disabled and no prefetech I get the same error. Have to wonder what Dimzon is.

[DTL]

"I don't use avstp."

But your script shows mt=true in every mvtools call. It is not required for AVS+ mt. It only switch on/off avstp-based internal mt.

"Have to wonder what Dimzon is."

Yes - it is strange error. 'dimzon' token is not found in AVS+ 3.7 sources. Can you run avsmeter from command line ? I test with avsmeter and virtualdub x64 (also encoding with x264 command line - x264 direct reading src.avs script file).

All files from archive is now required - the Compute.cso must be located in the same folder with mvtools2.dll. Its loading engine (some helper function from DX12 toolkit) mostly probably searches for file near mvtools2.dll in the same path only. It is compiled shader file. I do not know how to pack (link ?) in single executable resources if even possible. So all new versions with compute shaders will have a set of .cso files - some like SAD.cso, Mdegrain.cso and may be Msearch.cso and other. I even not sure if it good and possible to pack different compute shaders in the single file. In theory reader loads some bytestream from file so it possible to read some known parts start-end from single binary file but it is additional work and source of errors.

"Removing the optSearchOption=5"

It is disabling all DX12-based hardware accelerated processing and fallback to 'standard' mvtools 2.7.45 processing.

[magnetite]

Here's a link to the combined (https://i.imgur.com/KTYGIXb.png) error messages. The last DLL that worked for me was on 12/31/2021. The CSO file is in the same folder as the mvtools2 DLL.

[DTL]

"LSMASHVideoSource("E:\Mobile Video\MVI_0564.mp4")"

What is frame size ? Can you run with FullHD internal source like

ColorBarsHD(1920,1080) ?

Currently it looks the height must be integer divisible to block size so for FullHD progressive it is 1920x1080 and for interlaced - pad to 1920x1088 to get 1920x544 after SeparateFields() and so on.

"link to the combined error messages."

Evaluate: Unhandled C++ exception - it is AVS+ error message. I see you use still not 'final' 3.7.2 AVS+ build.

I see same error message at pinterf's github: https://github.com/pinterf/AviSynthPlus/issues/42 . May be not all more pointing to place of problem messages still displayed by Avisynth correctly ?

[magnetite]

The frame size of that video source was 1920x1080 progressive.

I still get the same error message with the official (https://github.com/AviSynth/AviSynthPlus/releases/tag/v3.7.1) Avisynth+ build from Github. Is that the final build you're referring to?

With ColorBarsHD as the source, same error message pops up.

[DTL]

"with the official Avisynth+ build from Github."

Yes - I also setup latest 'release' 3.7.1 at my the only working system with GTX1060. I think of creating debug build also it have enabled DirectX debug layer - it can be installed as part of Windows 10 user-side and may be will shows more detailed errors. It may be something around structured exceptions handling - may be it is disabled in release build so exception do not catched in the underlying layers and passed to AVS+ host.

Here is debug build - https://drive.google.com/file/d/1yENQ-_NN39jPdzo_bFrJJ5UWc__Q_o0U/view?usp=sharing . Also added exception handling around loading of Compute.cso file. It have builds with 2 types of C++ exception handling - EHa and EHsc in VS2019 settings - may it helps.

*updated link to archive with debug MS .dlls included*

[takla]

DTL
I get the following error "MAnalyse: Can not load file Compute.cso ReadData"
Any way to manually specify the file path for it? I tried mvtools2_EHa.dll & mvtools2_EHsc.dll but both give me this error.

[DTL]

Same was with magnetite user. Recommended: to put all files in the some very simple folder path like
c:\AVS\
(not something like c:\my_regional_letters_path\some folder Name\avisynth+\ )

Last message he report solve the problem but not exactly how.

Simple recommended setup:
put all files (mvtools2.dll, Depan.dll, Depanestimate.dll, Compute.cso, src.avs script, avsmeter64.exe) in path like
c:\AVS\
and run 'avsmeter64 src.avs' command from command line with current/working folder path c:\AVS (I use FAR file manager for 'navigate to some folder and run from it').

I will try to re-write the loading function of shader file in next builds - it looks C++ Microsoft-based sample file reading function have some issues with some paths at some user's Windows setups. There possible tons of different ways of load binary file data from file into RAM buf in C/C++/WinAPI etc.

May be loading issue is because of typically using path (for plugins) like C:\Avisynth+ with "+" sign in path name. I do not use this paths in my development windows so can not see error.

Currently used function is 'microsoft signed' :) https://github.com/microsoft/Xbox-ATG-Samples/blob/9654b6efa4e1db52365fc2ae80386e0e27b9658d/Kits/ATGTK/ReadData.h

Helper for loading binary data files from disk
//
// For Windows desktop apps, it looks for files in the same folder as the running EXE if
// it can't find them in the CWD

CWD is possibly Current Working Directory

May someone see some sources of error to fix in it ? The placement of EXE is looks like not an issue - I can load src.avs script in VirtualDub.exe located from very different path.

Simple try-catch structure looks like can not point to exact line where error happens:

// load compute shader
std::vector<uint8_t> computeShaderBlob;
try {
computeShaderBlob = DX::ReadData(L"Compute.cso");
}
catch (std::exception& e)
{
env->ThrowError(
"MAnalyse: Can not load file Compute.cso %s", e.what()
);
}


It looks e.what() return only 'ReadData' string.

[magnetite]

Same was with magnetite user. Recommended: to put all files in the some very simple folder path like
c:\AVS\
(not something like c:\my_regional_letters_path\some folder Name\avisynth+\ )

Last message he report solve the problem but not exactly how.


I uninstalled Avisynth+ from C:\Avisynth+ and reinstalled it to C:\AVS as suggested, then put everything in C:\AVS\plugins64. I deleted the plugins64+ folder just to be safe. VRAM usage was around 4 GB.

[DTL]

+ some note: as I later found that builds have 'reversed' scaleCSAD adjustment for DX12 processing (2 used as -2 for 'classic MAnalyse' and so on). The default 0 works as 0.
Missed the lines

// YV12 YV16 YV24
// nLogXRatioUV 1 1 0
// nLogYRatioUV 1 0 0
// effective_chromaSADscales: (shift right chromaSAD)
// chromaSADscale=0 -> 0 1 2 // default. YV12:no change. YV24: chroma SAD is divided by 4 (shift right 2)
// =1 -> -1 0 1 // YV12: shift right -1 (=left 1, =*2) YV24: divide by 2 (shift right 1)
// =2 -> -2 -1 0 // YV12: shift right -2 (=left 2, =*4) YV24: no change
effective_chromaSADscale = (2 - (nLogxRatioUV + nLogyRatioUV));
effective_chromaSADscale -= chromaSADscale; // user parameter to have larger magnitude for chroma SAD
// effective effective_chromaSADscale can be -2..2.
// when chromaSADscale is zero (default), effective_chromaSADscale is 0..2

of PlaneofBlocks.cpp in sending params data to shader.

nLogxRatioUV + nLogyRatioUV for YV12 is 1+1 so
effective_chromaSADscale -= chromaSADscale is
2 - (1+1) = 0 - chromaSADscale - that reverses adjustment.

Will be fixed in the next build.

[takla]

ColorBarsHD(1920, 1080)
ConvertToYV12()
tr = 3
super = MSuper (pel=1, levels=1, chroma=false)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=5, overlap=0, levels=1, chroma=false)
MDegrainN (super, multi_vec, tr, thSAD=150)
Prefetch(12)


Encoded 600 frames in 17.893s (FFV1)
GPU usage is around 8%. Very low.

CPU only takes 14.556s.

[DTL]

It can now work with chroma=true.


super = MSuper (pel=1, levels=1, chroma=true)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=5, overlap=0, levels=1, chroma=true)


The close 'software mode' is only


super = MSuper (pel=1, levels=0, chroma=true)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=1, overlap=0, levels=0, chroma=true)

[takla]

DTL
Using the script posted in #64, optSearchOption=5 is about ~3 seconds slower then optSearchOption=1.

[DTL]

"script posted in #64,"

Do not uses chroma processing in MAnalyse. That may make quality of motion search worse (and cause visible bugs in some cases - https://forum.doom9.org/showthread.php?p=1783361#post1783361 ) . So enabling chroma in MAnalyse generally increase quality (and recommended to use if speed is acceptable).

It may be interesting to check the speed lost of MAnalyse between

optSearchOption=1 chroma=true/false (onCPU only processing via host RAM)

and

optSearchOption=5 chroma=true/false (DX12_ME + CS via HWAcc RAM)

The ME hardware engine always uses chroma in search and enable/disable in MAnalyse settings only switch on/off UV planes copying from source to upload buffer and chroma SAD enable/disable compute and addition to output SAD in CS. So with optSearchOption=5 enabling chroma is close to free for speed. And with CPU processing enabling chroma in MAnalyse is typically very visible speed impact.

The second totally winner feature of HW ME - it uses always pel=4 search that is awfully slow onCPU. Though its full usage in mvtools still not implemented (may cause also very slow MDegrain onCPU). It will be used in SO=6 mode for increasing speed with blocksize=16. Using pel=4 precision may also makes degraining a bit better. It possibly can be tested with only optSearchOption=0 or may be 1 max.

"GPU usage is around 8%. Very low."

Is it Video Encode load % of 'GPU (total ?) %' ?

[takla]

It is total GPU load. And I wanted to compare apples to apples, that is why I had chroma set to false for both gpu and cpu tests.
I'm not too worried about quality at this stage, only wanted to test speed. But if the speed penalty for chroma=true is low with gpu, I'll keep that in mind for the future, thanks.

Edit: Tested chroma=true with GPU & CPU, CPU is still faster.

[DTL]

"It is total GPU load."

I hope you use for speed-test release build from github ? Not debug build.

"CPU is still faster."

It looks you use AMD CPU with large cache. For intel-users with low L2/L3 cache the MAnalyse may be more slow.

"Tested chroma=true with GPU & CPU"

What is speed difference of chroma=false and chroma=true for MAnalyse with optSearchOption=5 ? And what GPU-card is used ? With chroma=true in MSuper (so it is enabled for MDegrain).

[takla]

I didn't think debug build would make a difference. My bad. Here are my new tests, again, with the settings used in post #64:


r.2.7.46-a.06
time=15.841s
MSuper & MAnalyse chroma=true
optSearchOption=5

r.2.7.46-a.06
time=15.554s
MSuper & MAnalyse chroma=false
optSearchOption=5

r.2.7.46-a.06
time=4.016s
MSuper & MAnalyse chroma=true
optSearchOption=1

r.2.7.46-a.06
time=3.576s
MSuper & MAnalyse chroma=false
optSearchOption=1

CPU: AMD Ryzen 9 3900x
GPU: AMD Radeon RX 5700


I didn't think the difference would be THAT much. Also, Video Codec usage under GPU in task manager showed 99% usage. So again, my bad on that.

CPU is almost 4 times faster.

[DTL]

"I didn't think debug build would make a difference."

It is full of software checks and very slow. Only provided to get possibly more pointing error message about that C++ exception issue at user-side.

" AMD Radeon RX 5700"

Thank you for information. It is really the first report of non-NVIDIA accelerator usage.

The good news that AMD also provides drivers for DX12_ME operation. And no need for any software tuning between AMD and NVIDIA products. It was expected. Good work for compatibility from Microsoft. The other news is that Ryzen 9 3900x is about 4x faster. Though I have not any info about expected speed of Video Encoder engine in AMD accelerators (if it even exist there). I think it will be some future products from Intel and AMD that will provide hardware features for DX12_ME operation.

Also as expected the speed difference between chroma true and false with accelerator is small enough - about 2%. With 'onCPU' processing is about 12% (still not very large - but it is AMD large-cache device or too small frame size for this cache size).

Update: Known issue and workaround: Lower up to 8 rows of blocks (with block size 8x8) may be somhow not correctly processed at some scenes. Current solution: pad bottom of 1920x1080 frame with 72 lines with AddBorders() and Crop() after processing.
Example:
Addborders(0,0,0,72)
...msuper, manalyse, mdegrain..
Crop(0,0,0,1080)

[takla]

Update: Known issue and workaround: Lower up to 8 rows of blocks (with block size 8x8) may be somhow not correctly processed at some scenes. Current solution: pad bottom of 1920x1080 frame with 72 lines with AddBorders() and Crop() after processing.
Example:
Addborders(0,0,0,72)
...msuper, manalyse, mdegrain..
Crop(0,0,0,1080)
Mhh that is messy. Can't you make mvtools do exactly that, but internally?

[DTL]

It is not long time solution I hope. Just a known way of bugfix *if* it occur. I even not sure where is it come from. Most of buffers copying-processing is totally static but issue happens rarely (may be 1% of frames or less) on some moving patterns near bottom of frame. Like water waves or some camera pan or objects move. May be it even issue of my NVIDIA GTX1060 NVENC (with current driver, etc).
Or it may be SAD compute shader threading dispatch (may not process last threads groups sometime ?) issue. But its Dispatch() call also equal for any frame so the issue should be visible in any frame like no denoising (?) currently if it cause not same blocks blending it may return small or close to zero SAD for several bottom rows of blocks ?

Comment from Microsoft support about used file loading function:

The DX::ReadData helper looks in the current working directory at runtime for the file.

When building for Win32 desktop, if it can't find it there, then it looks in the same folder as the running EXE. If you are a 'plugin' then likely that directory where you have your plug in installed.

Your best option is to use some full path to your "plugin" directory to find your data and not rely on CWD.

So it looks I need to look how 'avstp.dll' is searched for possible paths for more use cases of avisynth + plugins + scripts paths installation and running. It looks ReadData() can not found compute.cso file if mvtools.dll is copied in some path not equal to loaded script path ? I usually put all files in 1 folder (source file, all used plugins, source script) so do not see this issue.

[DTL]

New update: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.07

Now pel=4 have about usable speed. At i5-9600K + GTX1060 with x264 encoding of 1080i footage SO1 is about 3 times slower in compare with SO5 using pel=4. About 1.26 fps vs 3.76 fps. Pel=1 works with 6.46 fps SO=5.
And it is still onCPU SAD computation for pel 2 and pel 4. Next versions with onHWAcc SAD computation for pel 2 and pel 4 expected to be even faster.
The quality at some quick tested camera pan movements with pel=4 is a bit better in compare with pel=1.

Example of processing script:

tr=25
super=MSuper(last, mt=false, chroma=true, pel=4, hpad=8, vpad=8, levels=1)
multi_vec=MAnalyse (super, multi=true, blksize=8, delta=tr, overlap=0, chroma=true, optSearchOption=5, mt=false, levels=1, scaleCSAD=2)
MDegrainN(last,super, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16)


MDegrainN new added params:
1. adjSADzeromv (1.0 - default, no op) - possible SAD multiplier for zero-move blocks (before thSAD processing for getting block's weighting value). Float value. Recommended values: 0.9..0.4-. Possible medium values 0.75..0.5. Allow to increase degraining at static areas.

Example: setting 0.5 result thSAD for zero move blocks (static) will be thSAD*2.

2. adjSADcohmv (1.0 - default, no op) - possible SAD multiplier for blocks in coherent moving areas (before thSAD processing). Float value. Recommended values: 0.9..0.4-. Possible medium values 0.75..0.5. Allow to increase degraining at big enough coherent moving areas of much larger 1 block_size size (like camera pan movement over non-changing scene).

3. thCohMV (-1 default, no op) - threshold to detect if block's move vector is equal to surround blocks (top,left,right, down) move vectors. -1 - disables this part of processing (faster), 0 - lowest working value. Recommended values 0..4. Possible range - 0..unlimited int. Too high values will create error-blended blocks (like with too high thSAD value).

The thCohMV is not auto-scaled with setting pel to 2 or 4 and as integer MVs length become 2 and 4 times larger it looks also required correction of thCohMV param about xPel. I do not think internal auto-correction is good because in integer param it will limit range of fine-tuning. Or the integer need to be chenged to float.

Also with pel 2 and pel 4 it looks good to add 1 more param like thZeroMV - threshold to detect if block is close to zero move. With pel > 1 currently even the small sub-pel movements is treated as non-static block. Currently large areas of 'static' blocks also may be adjusted with adjSADcohmv param that can be applied to blocks non-coherency movement (including deltas around zero) defined by thCohMV param.

[takla]

DTL what source filter do you use? I tried both, LWLibavVideoSource and AviSource but they both crash without error.
The only way I can test optSearchOption=5 is with ColorBars.

Anyways, here are some more benchmarks:

ColorBarsHD(1920, 1080)
ConvertToYV12()
Trim(0, 600)
tr = 3
super = MSuper (pel=1, levels=1, chroma=true)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=1, overlap=0, levels=1, chroma=true)
MDegrainN (super, multi_vec, tr, thSAD=150)
Prefetch(12)

r.2.7.46-a.07

ffmpeg -y -benchmark -i TEST.avs -c:v ffv1 TEST.mkv


optSearchOption=5

pel=1 chroma=false
time=14.972s

pel=1 chroma=true
time=15.236s

pel=2 chroma=true
time=15.710s

pel=4 chroma=true
time=15.910s



optSearchOption=1

pel=1 chroma=false
time=3.695s

pel=1 chroma=true
time=4.091s

pel=2 chroma=true
time=4.807s

pel=4 chroma=true
time=5.487s

[DTL]

I use FFMpegSource2() from ffms2.dll. The current builds very critical to V size of frame. Known working sizes 1920*1080 (progressive). 1920*1088 (interlaced for SeparateFields(), 1920*1152 (1080+72) to workaround possible low rows issues and interlaced frame. Still not search in deep what is limit size or how is workaround it inside. But with non-compatible size it throws error like 'error - commandlist-close' or other text error. It looks typical working height (for interlaced source) is integer divisible to 16.

I see you use tr=3 with low params MDegrainN - that cause fallback to MDegrain3 and not tested. It may not work because now MDegrainN uses different way of loading MVs from 'fake' structures. The default creation of 'Fake*' class is to use old scattering of MVs data into FakeBlock class. And will cause crash with MDegrain(X) trying to load from uninitialized memory.
Also SO=5 is not good compatible with MDegrain(X) because their block-procesing functions do not have additional MVs limiting of invalid vectors (sometime coming from hardware ME). With invalid (far outside frame area) MVs the MDegrain(X) will crash without good error message.
Currently only pel=2 and pel=4 with old MAnalyse SAD computation should be compatible with MDegrain(X). The MAnalyse do additional check and limiting of MVs before SAD computing. May be it is good idea to put this limiting into MAnalyse at MVs data coping function from readback buffer to MAnalyse output buffer.

To use MDegrainN with low (any) tr you need to set both thSAD and thSAD2 to different values like 150 and 150-1. To make editing easier it may be created script user-variable like my_thSAD and set it to MDegrainN as

my_thSAD=150
MDegrainN(thSAD=my_thSAD, thSAD2=my_thSAD-1)


Benchmarks shows your CPU is great with core speed and cache size and may be performance of DX12ME at accelerator is limited to 15-sec with tr=3. With 4k frame and larger tr when frameset with pel4 will not fit in CPU cache the onCPU speed may be lower.

[takla]

DTL Specifying a value for thSAD2 is what fixed it, thanks.

LWLibavVideoSource("C:\Users\Admin\Documents\01.mkv")
Trim(0, 600)
Addborders(0,0,0,72)
tr = 8
super = MSuper (pel=4, levels=1, chroma=true)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=5, overlap=0, levels=1, chroma=true)
MDegrainN (super, multi_vec, tr, thSAD=150, thSAD2=75)
Crop(0,0,0,1080)
Prefetch(12)

r.2.7.46-a.07

optSearchOption=5

pel=1 chroma=false
time=42.293s
time=41.430s
262.419 KB

pel=1 chroma=true
time=41.253s
time=41.285s
246.503 KB

pel=2 chroma=true
time=45.720s
time=44.713s
248.503 KB

pel=4 chroma=true
time=41.868s
time=44.675s
247.812 KB

optSearchOption=1

pel=1 chroma=false
time=8.963s
265.645 KB

pel=1 chroma=true
time=9.085s
249.453 KB

pel=2 chroma=true
time=14.969s
251.571 KB

pel=4 chroma=true
time=21.300s
248.394 KB

File size was just for me to check if settings were used properly.
optSearchOption=5 seems to sometimes fluctuate by 3 seconds.
optSearchOption=1 is still much faster. I'll test some 4K.

Edit: Here is 600 4K frames with TR=6:
optSearchOption=1 time=205.274s
optSearchOption=5 time=211.290s
Or about ~3FPS.

[Boulder]

I'd like to test the new version, but I'm having a hard time following the complex explanations.

I.e. could someone please simplify things a bit by telling me which settings would make the result as close to a vanilla MVTools2 one as possible, in order to make a fair comparison? My normal chain of operation is MSuper (separate for analysis and MDegrain) - MAnalyse - MRecalculate (possibly two times with halving blocksize with each iteration) - standard MDegrain.

[DTL]

"4K frames with TR=6:
optSearchOption=1 time=205.274s
optSearchOption=5 time=211.290s"

With TR=12 the speed difference may finally reverse to SO=5 is better.

I got an idea how to make at least MDegrainN onCPU better speed for pel >1 still not going into full onHWAcc processing. So onCPU times of MDegrainN part of processing will be lower.
It is about not reading sub-pel shifted copies of frame from memory but shift single block to sub-pel before averaging inside CPU register file or caches. At least for MDegrainN with 1 src block using only 1 ref shifted-position block it should make better speed. And with SO=5 the MVs creation is now serviced with dedicated accelerator. The idea come from the SAD calculation for pel >1 : I not like an idea to upload to accelerator 4 for pel=2 or 16 for pel=4 copies of the ref frames produced by MSuper(). It is slower with upload via bus and wastes fast but limited in size RAM of accelerator awfully. So I will re-make current SAD compute shader to make sub-pel shift of ref block from single full-size ref frame inside shader before SAD calcultaion. It is natural enough task for data compute accelerator.

" My normal chain of operation is MSuper (separate for analysis and MDegrain) - MAnalyse - MRecalculate (possibly two times with halving blocksize with each iteration) - standard MDegrain."

May be better to post your current script so when someone will have time - will tried to change its MAnalyse processing to SO=5 and look if it work / if it possible. Also the input frame size is required.

[Boulder]

" My normal chain of operation is MSuper (separate for analysis and MDegrain) - MAnalyse - MRecalculate (possibly two times with halving blocksize with each iteration) - standard MDegrain."

May be better to post your current script so when someone will have time - will tried to change its MAnalyse processing to SO=5 and look if it work / if it possible. Also the input frame size is required.

It's basically something like this. The last MRecalculate iteration is often skipped for 4K so it's just MAnalyse+MRecalculate before MDegrain.

source("whatever.dgi") # 4K or 1080p

blksize = 16 # or 32 for 4K
blksize2 = 8 # or 16 for 4K
blksize3 = 4 # or 8 for 4K
overlap = 8 # or 16 for 4K
overlap2 = 4 # or 8 for 4K
overlap3 = 2 # or 4 for 4K

prefilt = ex_minblur(r=1, uv=3)
superanalyse = prefilt.msuper(pel=4, sharp=1, rfilter=2)
supermdg = msuper(pel=4, levels=1, sharp=1, rfilter=2)

fv1 = manalyse(superanalyse, isb=false, delta=1, blksize=blksize, overlap=overlap, search=4, searchparam=2, pelsearch=2, truemotion=false, dct=5, scalecsad=2, trymany=false, global=true)
bv1 = manalyse(superanalyse, isb=true, delta=1, blksize=blksize, overlap=overlap, search=4, searchparam=2, pelsearch=2, truemotion=false, dct=5, scalecsad=2, trymany=false, global=true)
fv1 = mrecalculate(superanalyse, fv1, thsad=100, blksize=blksize2, overlap=overlap2, search=4, searchparam=2, truemotion=false, dct=5, scalecsad=2)
bv1 = mrecalculate(superanalyse, bv1, thsad=100, blksize=blksize2, overlap=overlap2, search=4, searchparam=2, truemotion=false, dct=5, scalecsad=2)
fv1 = mrecalculate(superanalyse, fv1, thsad=100, blksize=blksize3, overlap=overlap3, search=4, searchparam=2, truemotion=false, dct=5, scalecsad=2)
bv1 = mrecalculate(superanalyse, bv1, thsad=100, blksize=blksize3, overlap=overlap3, search=4, searchparam=2, truemotion=false, dct=5, scalecsad=2)

convertbits(16).mdegrain1(supermdg, bv1, fv1, thsad=150, thsadc=200, thscd1=500, thscd2=90, limit=0.2, limitc=0.45)

[takla]

With TR=12 the speed difference may finally reverse to SO=5 is better
DTL At 4K and TR=6 around 10 to 12GB of my RAM is being used. But at TR=8 I have to cancel the encoding because 99% of my RAM (32GB) is used. (Possible memory leak? Why does ram usage increase so much?) So any higher TR at 4K is not possible.

Boulder there have been plenty of examples in this thread. See post #77.

[DTL]

"Why does ram usage increase so much?"

Pel=4 finest level is currently 16x more RAM in compare with pel=1 for 'super' clip (+lower sized levels) and it is multiplied to number of AVS+ threads and to AVS+ cache system. So it looks old developers not went down to pel=8 with 64x more RAM usage.

Each 'super' frame for pel=4 and 4K is about 2160x17 = about 37000 in height size. You can check it - return 'super' clip and see its frame size.

So may be ask AVS+ support how to decrease cached frames by AVS+ ? I read somewhere about 2-params Prefetch() - like

Prefetch(N, M)

where one value is num of threads and second is cached num frames ? Do not found anything about it in docs. Also there are 2 values of cache control:
SetCacheMode(mode)
AVS+Fine tunes the internal frame caching strategy in AviSynth+.
Available values:
0 or CACHE_FAST_START start up time and size balanced mode (default)
1 or CACHE_OPTIMAL_SIZE slow start up but optimal speed and cache size

May be try to set CACHE_OPTIMAL_SIZE ?

Also http://avisynth.nl/index.php/MT_modes_explained - may be adjusting of MT mode may decrease number of cached frames ?

I run tr=25 with 1080i at 16 GB system with few enough RAM usage - may be about half.

"Possible memory leak?"

It typically increases over time. I run 3 hours transcoding without leakage issues.

The 'fully optimized' mvtools with DX12_ME search and all pel modes on-shader SAD caculation and at-processing sub-shift MDegrainN will use 16x less host RAM for pel=4 processing.

[DTL]

ColorBarsHD(1920, 1080)
ConvertToYV12()
tr = 3
super = MSuper (pel=1, levels=1, chroma=false)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=5, overlap=0, levels=1, chroma=false)
MDegrainN (super, multi_vec, tr, thSAD=150)
Prefetch(12)


Encoded 600 frames in 17.893s (FFV1)
GPU usage is around 8%. Very low.



It looks I found where the DirectCompute load graph is shown: In Win 10 task manager 'GPU' window the hardware load graphs can be switched to Compute_0 and Compute_1. It looks all ComputeShader load is displayed only in these graphs. But some software may display sum load of many graphs (3d+cuda+compute+copy+videoencode+videodecode+...).
I still do not found what difference between Compute_0 and Compute_1 load graphs. At degrain running I see some about equal load in both graphs. But it finally not 0..1% load as displayed in 3D GPU graph.

Currently at some still in-progress version of sub-pel shifting for pel=2 and pel=4 for SAD computing I got about 35..40% load of both Compute_0 and Compute_1 graphs. At GTX1060 card and 1920x1080 interlaced processing (at about 18 fps output to AVSmeter). Not very few but still have some space to add MDegrainN in the future. Also I hope inside HWAcc the shifted (subpel-motion compensated) blocks may be reused in both SAD and MDegrainN computation.

Currently was and idea to download set of sub-shifted-blocks to host memory to send to MDegrainN but it will load memory transfer and still complex enough. It may be better to add onCPU subshift to MDegrainN as intermediate solution before transferring all processing to accelerator.

[DTL]

Somehow working update https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.08 . Now all supported pel values processed inside accelerator. Not sure if it good for pel 1 and 2 for fast CPUs but still no user-side selection where to process. For testing of onCPU pel 2 and pel 4 processing the previous version may be used.
Not very much tested for quality of SAD generation for pel 2 and 4 and it may more or less be different from onCPU old processing because of using different sub-sample shifting kernel.
It looks making compute shaders at HLSL is not very efficient because 'compute_X' load is significant now at GTX1060 card at relatively simple operation of runtime shifting of blocks with about 8 total samples kernel. Though the performance now is not depend on sub-pel shift value (can support any float shift with equal speed) and depend slightly only on kernel size (with half size of 4 the speed is a bit better, but still no user-control param and only separate build with internal constant define possible for now). So it may be mostly benefitical at pel=4 (depending on balance of CPU and accelerator speed).

[magnetite]

I think I went from 30% load up to 60% with this new build on my GTX 1080 Ti.

[DTL]

Some strategic idea: if sub-sample shifting takes significant time (resources) it can be somehow reused for degraining. Either to decrease host memory read traffic or to save time of host CPU from performing same shift operation one more time.

Possible ways:
1. Finish degraining inside accelerator (as planned). The already found limitation (at least for CS 5.1 standards): HLSL compiler reports about 16384 max recommended temp array size in threads group. It looks the limitation of 'register file' size of one core in accelerator (of some generation) ? So in current version of shader the number of threads in group was reduced to 4x4 to stop compiler from warning. The compiler allow to have more buf but warn about degrading performance (it can auto-offload temp array to main memory ?). And each thread currently have only small enough buf about 3x time block size to hold sub-shifted block (H and HV shifted).
But MDegrainN operation require to hold a set of 2_x_tr ref blocks to compute sad -> next compute weights from sads -> normalize weights and use blocks in averaging. Unfortunately it looks additive accumulation of shifted blocks in single temp buf is not possible. Because to get weight of block in the sum - we need to calculate all weights and normalize.
So it looks the sub-shifted blocks can not be stored in on-chip memory and need to be temporarily written to accelerator's main memory (it is typically faster in compare with host but usually not very for medium consumer accelerators). But this approach will limit more available memory in accelerator (need to store both source frames + shifted copies) and limit possible max tr-value.

2. Pack sub-shifted blocks into some framebuf and download to host memory and use as source for MDegrainN (instead of super clip of 4x or 16x time larger for pel 2/4). It will be close to current output of MCompensate I think. Though it will be new datastream for mvtools (like replacing of 'super' input clip in MDegrain arguments). And download operation from accelerator and loading into host cpu may takes some time. It allow to use any tr-value because not need to store all source + shifted frames in the accelerator's limited memory.

Also I still not sure if HLSL compiler make best possible asm program to compile convolution - may be some hand-crafted asm (inline-asm if possible in HLSL ?) may be faster and take less resources of accelerator. Need to read more how compute units in accelerator are designed. May be it sort of SIMD dispatch ports and can compute FMA of several floats per clock. Still not check what current HLSL compiler produce. It can output asm file but I need to read about its syntax and application to execution units in shader compute model.

[DTL]

Some first working example of low pass motion vectors internal filtering before MDegrainN processing: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.09

It is not final filter and not perfect - just first working example without significant output bugs. Currently only luma SAD is checked after new vectors calculated and compared with thSAD. If new filtered vector have SAD above thSAD - the original vector from MAnalyse is used.

The initial idea and issue about converting of noise (luma+chroma) into spatial (phase) noise at MDegrainN processing with nosied vectors was described at post: https://forum.doom9.org/showthread.php?p=1963966#post1963966

New control params for MDegrainN: MVLPFCutoff, thMVLPFCorr.

MVLPFCutoff: cut off frequency of the low pass filter for motion vector's components (dx,dy) in temporal (tr) axis.
Default 1.0 additional processing disabled.
Valid range 0.0..1.0. Estimated working range when enabled: 0.05 to 0.5. Values below 0.05..0.01 possibly change nothing because internal kernel size of filter is fixed 10 taps now.

thMVLPFCorr: Maximum difference between original and filtered vector's dx,dy components (any of component) for correction. If difference above this value (not internally scaled to pel value) - the original vector from MAnalyse is used.
Value =0 (default) disables correction completely (no LPF-processing effect even with MVLPFCutoff < 1.0). May be useful to fix some bugs at the footage with lots of different movement and noise.
Expected good value: pel*(4..10). It is mostly additional 'fail safe' limit. If no issues found it may be set to infinite (like frame_width * pel) to allow processing of very fast movements. Typical real upper value: about maximum inter-frame shift of moving subjects * 1.5 * pel.

Current production degrain script used for testing (interlaced 1080 source):

SetFilterMTMode("DEFAULT_MT_MODE", 3)

__source_here___

AddBorders(0,0,0,72)
ConvertToYV12(interlaced=true)
SeparateFields()

tr=15
super=MSuper(last, mt=false, chroma=true, pel=4, hpad=8, vpad=8, levels=1)
multi_vec=MAnalyse (super, multi=true, blksize=8, delta=tr, search=3, searchparam=2, overlap=0, chroma=true, optSearchOption=5, mt=false, levels=1, scaleCSAD=0)
MDegrainN(last,super, multi_vec, tr, thSAD=185, thSAD2=170, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFCutoff=0.1, thMVLPFCorr=50)

Weave()
Crop(0,0,0,1080)


Support of overlap processing using filtered vectors still not implemented. It is not complex but need some time.

Found and fixed some point of memory leak in MDegrainN - may be it adds to the issues on February builds too.

[DTL]

Some more idea: typically 'non-simple' degrain scripts use some pre-denoised pre-processed clip as source for MAnalyse (super clip for MAnalyse). Sometime the preprocessing is as simple as some low-pass filtering like blur. So the idea is to add this simple pre-processing into MAnalyse with hardware search options to offload more work to accelerator. The processing may be done with compute shader dispatched with uploaded to accelerator frames before sending to ME engine. It will free more host resources for MPEG encoding.

[Dogway]

That's a good idea. The 'standard' though is to use MinBlur() which denoises more flat areas and less edge areas. A 'cheap' alternative similar to MinBlur is to use Inter Quartile Median (IQM), that would be easier to implement.

[DTL]

I see typical 'pre-filter' in QTGMC is


prefilt = last
w = prefilt.width()
h = prefilt.height()
removegrain(12, 12).gaussresize(w, h, 0, 0, w+0.0001, h+0.0001, p=2).mergeluma(prefilt, 0.1)


That is equal in result (may be not speed) to

Blur(1).gaussresize(w, h, 0, 0, w+0.0001, h+0.0001, p=2).mergeluma(prefilt, 0.1)

Where combination of Blur() and gaussresize() is 2 low-pass filters in a sequence (may be merged to single with combined transfer characteristic).
That is all internal AVS+ processing operators. May be simplified to SomeLowPassFilter(args).mergeluma(prefilt, 0.1) that mean mixing input plane with weight 0.1 to low-pass filtered plane.

In SMDegrain script I see much more complex pre-filter processing.

[DTL]

Trying to simulate 'overlap' processing with non-overlap MAnalyse/MDegrain I try to make 2 processing paths with half-blocksize diagonal shift:

BkSz=8
BkSz_d2=BkSz/2

AddBorders(BkSz_d2,BkSz_d2,BkSz_d2,BkSz_d2)

no_sh=last
sh=Crop(BkSz_d2,BkSz_d2,width-BkSz_d2, height-BkSz_d2).AddBorders(0,0,BkSz_d2,BkSz_d2)

tr = 12 # Temporal radius
super_no_sh = MSuper (no_sh, chroma=true, pel=2)
super_sh = MSuper (sh, chroma=true, pel=2)

multi_vec_no_sh = MAnalyse (super_no_sh, multi=true, chroma=true, overlap=0, search=3, searchparam=2, delta=tr, mt=false, optSearchOption=1)
multi_vec_sh = MAnalyse (super_sh, multi=true, chroma=true, overlap=0, search=3, searchparam=2, delta=tr, mt=false, optSearchOption=1)

no_sh=MDegrainN(super_no_sh, multi_vec_no_sh, tr, thSAD=200, thSAD2=190, mt=false)
sh=MDegrainN (sh, super_sh, multi_vec_sh, tr, thSAD=200, thSAD2=190, mt=false)

#back
sh=AddBorders(sh, BkSz_d2,BkSz_d2,0,0).Crop(0,0,width-BkSz_d2, height-BkSz_d2)

Layer(no_sh, sh, "fast")

Crop(BkSz_d2,BkSz_d2,width-BkSz,height-BkSz)


The result looks really better in compare with no-overlap processing but still not as smooth as overlap=blocksize/2. Also the speed is about 2x better in compare with overlap=blocksize/2 (with CPU only processing).

Edit: may be right solution is not about deblocking of layers but creating correct blending mask. Like rhomb-shaped per each block tiled over all blocks of frame. Will try it with auto-sizing of mask using 'for' loops of AVS+.

[DTL]

New version: https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-a.10

Added MVLPFGauss MVs low-pass filtering mode to MDegrainN as single control-param adjustment. MVLPF implemented in all processing modes of MDegrainN (chroma enabled and overlap enabled). Default = 0 (disabled), float param. Expected practical adjustment range 0.5..3.0. Too low values like <0.1 will mostly disable procesing, too high like 10 may cause bugs because internal convolution kernel is about 10 samples size and too high sigma values will make kernel not gauss-shaped but rectangular. The old values of 2-params LPF MVLPFCutoff and MVLPFSlope still exist but suspended to future development because simple gauss-kernel for LPF looks produce good results already. The speed of processing should not depend on kernel type. The non-over/undershoot gauss-kernel processing expected to make good results and the development of other LPF with controlled both cut-off frequency and slope with non-over/undershoot performance is more complex. Left for the future versions.

Added usage of scaleCSAD param defined in MAnalyse in the secondary SAD check after MVLPF processing in MDegrainN.

Fixed bug in MAnalyse that cause random processing aborting with error message 'motion vectors clip too short' (in SO=5). It magically works with non-initialized memory long time in the past but start to fail frequently with >1 MAnalyse in the script or testing with single pair of src-ref frame and MShow().
Fixed bugs in SAD computation in shader with luma and chroma with pel=2 and with chroma with pel=4.

When experimenting with shifted layers blending for simulate overlap processing I found the small padding of the single layer with about blocksize/2 also makes output MPEG encoded speed a bit lower. May be it is about interacting of previous MPEG compressed source blocks tesselation with hardware MVs search engine blocks tesselation. So current single layer processing script for 1080i source is:

SetFilterMTMode("DEFAULT_MT_MODE", 3)

__source_here___

AddBorders(0,0,0,72)

SeparateFields()

BkSz=8
BkSz_d2=BkSz/2
AddBorders(BkSz_d2,BkSz_d2,BkSz_d2,BkSz_d2)

tr = 15 # Temporal radius
super = MSuper (chroma=true, pel=4, levels=1)

multi_vec = MAnalyse(super, multi=true, blksize=8, delta=tr, overlap=0, chroma=true, optSearchOption=5, mt=false, levels=1)
MDegrainN(super, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9, thMVLPFCorr=100)

Crop(BkSz_d2,BkSz_d2,width-BkSz,height-BkSz)
Weave()
Crop(0,0,0,1080)


It looks current float-based sub-shifting in the shader (for pel=2 and pel=4) implementation not good in speed and need to make some integer-based implementation to see if it will be faster.

Some number of tests with pre-filtering of clip for MAnalyse (with 'simple' processing like from QTGMC) still shows close to no improvement in output MPEG output speed. May be current additional low-pass filtering of MVs in time domain before MDegrain works in the close way to low-pass filtering in spatial domain before MAnalyse.

All new features of MDegrainN also works with software modes of MAnalyse so not require Win10+DX12-ME hardware for usage so applicable to https://forum.doom9.org/showthread.php?t=173356 thread mvtools.

[takla]

Here are my thoughts so far:

I don't see a point in using GPU acceleration at all (unless you somehow manage to make it SIGNIFICANT faster to what it is now) because the only time GPU is faster right now, is in niche cases like TR=>10 with pel=2 or 4 at 4K. But even in those cases you need over 64GB of system RAM or else you run out of it, as I've pointed out before.

And considering hardware "tiers", when looking at the same price class, a $500 CPU will easily beat a $500 GPU here.

So personally I'd prefer improvements on the CPU side of things, because it makes more sense.

[tormento]

I don't see a point in using GPU acceleration at all
Because not everybody owns a 16 cores CPU and whatever resource you can free, it can be allocated to other filters or encoding.

[takla]

Because not everybody owns a 16 cores CPU and whatever resource you can free, it can be allocated to other filters or encoding.

Good point.

[DTL]

" the only time GPU is faster right now, is in niche cases like TR=>10 with pel=2 or 4 at 4K. But even in those cases you need over 64GB of system RAM or else you run out of it, as I've pointed out before."

At my current work setup of old enough i5-9600K CPU + GTX1060 accelerator and transcoding with x264 with close to 'placebo' settings I got about 3.5 fps with CPU only and about 5+ fps with SO=5 option. So with hardware acceleration I can process more footages per work day. The blockiness artifacts without overlap is very rare at that footage - mostly on large size fire flames or large size smoke. The water looks good.

Also I still not show here the tests but the 'star-like' hyperbolic zoneplate sub-sample moving + noise test shows a bit better motion compensation in V-direction (horizontal part of hyperbolic zoneplate) in compare with internal MAnalyse pel=4 search. Will try to post comparison results next time when will be at work.

The interlaced 1080 runs well with 6 threads at 16 GB Win10 system. Takes about 50% of RAM, so I expect 4K will takes about 32 GB at 6 threads (with 'typical' AVS+ cache control). But if you run at massive multicore CPU with >10 cores it really can overflow 64 GB RAM. So it may be good to limit number of threads per AVS and leave some free cores to MPEG encoder only.

"will easily beat a $500 GPU here."

I expect when prices after mining will drop - the old enough accelerators much cheaper $500 will be good to installed as 1 or 2 (or more) per host to help free CPU resources to MPEG encoding. Also the >1 accelerator I hope can be used for 'overlap' simulation using either internal AVS scripting (with masked Overlay() function) or I found old Fizick's plugin BlockOverlap with internal mask generation and blending 2 half-blocksize diagonally shifter layers. Though it is C-only and may be not as good in speed as possible AVS+ internal Overlay() filter. The mask for Overlay may be loaded from 8x8 BMP file and tiled using scripting to required frame size. Also this method allow to use any handcrafted blocks blending mask in MSPaint or any other pixel-setting editor. Or may be some scripting-way is possible to create clip of typical blocksize 8x8 with perset samples values.

Though currently I have only about 30% of VideoEncoder load - may be it is possible to make some more advanced mod of MAnalyse to send 2 pairs of frames per command sequence and get 2 ME output results for 'special' mode of MAnalyse for alternative motion-clip format for overlap blending (with not same blocks positions as with 'old overlap mode of MAnalyse+MDegrain) and special mode in MDegrainN for alternative overlap blending. It should be best in speed but require more programming work. It may be already tested for quality using AVS scripting.

The diagonal shift of 'shifted' version of clip in MAnalyse to send to ME accelerator is very easy - just shift starting address of buffer reading at creation of 'upload' command (and after typical padding after MSuper it will not buffer-overrun at the lower-right corner of buffer). In scripting it require combination of AddBorders+Crop that may produce much more memory bus traffic.

I make x64 build of Fizick's BlockOverlap plugin for new AVS+, but still not test it yet: https://github.com/DTL2020/BlockOverlap

"prefer improvements on the CPU side of things"

It is also planned. Currently in progress the internal (inside CPU) shifting of blocks for MDegrain for pel > 1. So with SO=5 in MAnalyse it can completely skip larger sub-shifted planes creation and decrease memory bus traffic. Currently the latest build have not finally debugged 'tech speed test demo' of this mode - new option for MSuper(pelrefine=false) to disable pel >1 planes creation and MDegrainN(UseSubShift=1) to enable alternative request of sub-shifted block from Fake* structure. It currenty have only integer AVX2 implementation for block size 8x8 (and luma only) but the AVX512 may be a bit faster. Currently not in production state - create distorted output but perform the full processing. Hope to look into debug of it soon.

It also will greatly decrease amount of memory for 'super' clips - about 15 times less for pel=4. Currently it is not made because it require to change size of 'super' clip and look if it not crash the underlying processing somewhere. Currently only CPU load disabled for pelrefine=false MSuper mode to test speed benefit that is much easier. So after this change will be finished and 'super' clip will be cropped to only 1x frame size we will get large memory saving with pel=2 (about 3 times) and pel=4 (about 15 times) with hardware ME modes. I theory in that case the usage of 'super' clip in that case will be mostly eliminated (may be to store padded 1x frame only with levels=1).

Also using same ideas of internal scaling of patch in CPU register file planned to make MAnalyse search optimized for pel >1 in same way. But it require more complex SIMD programming.

[tormento]

It currenty have only integer AVX2 implementation for block size 8x8 (and luma only) but the AVX512 may be a bit faster.
Please, keep in mind that someone (me) still has AVX CPUs. :o

[DTL]

AVX mean it do not have fast enough integer operations with increased size register file. Also operations are limited to SSE2 128bit integer per op, AVX2 allow 256bit integer ops that is virtually twice faster. It is better to upgrade to AVX2 CPU at least in 202x years. Intel promises AVX1024 in the mid of 202x already.

[kedautinh12]

Hi, but anyone have much money like you :D

[tormento]

Intel promises AVX1024 in the mid of 202x already.
Perhaps on Xeons. They are disabling AVX512 in consumer CPUs.