New DirectX 12 Video APIs

[DTL]

Same was with magnetite user. Recommended: to put all files in the some very simple folder path like
c:\AVS\
(not something like c:\my_regional_letters_path\some folder Name\avisynth+\ )

Last message he report solve the problem but not exactly how.

Simple recommended setup:
put all files (mvtools2.dll, Depan.dll, Depanestimate.dll, Compute.cso, src.avs script, avsmeter64.exe) in path like
c:\AVS\
and run 'avsmeter64 src.avs' command from command line with current/working folder path c:\AVS (I use FAR file manager for 'navigate to some folder and run from it').

I will try to re-write the loading function of shader file in next builds - it looks C++ Microsoft-based sample file reading function have some issues with some paths at some user's Windows setups. There possible tons of different ways of load binary file data from file into RAM buf in C/C++/WinAPI etc.

May be loading issue is because of typically using path (for plugins) like C:\Avisynth+ with "+" sign in path name. I do not use this paths in my development windows so can not see error.

Currently used function is 'microsoft signed' https://github.com/microsoft/Xbox-AT...GTK/ReadData.h

Helper for loading binary data files from disk
//
// For Windows desktop apps, it looks for files in the same folder as the running EXE if
// it can't find them in the CWD

CWD is possibly Current Working Directory

May someone see some sources of error to fix in it ? The placement of EXE is looks like not an issue - I can load src.avs script in VirtualDub.exe located from very different path.

Simple try-catch structure looks like can not point to exact line where error happens:

  // load compute shader
  std::vector<uint8_t> computeShaderBlob;
  try {
    computeShaderBlob = DX::ReadData(L"Compute.cso");
  }
  catch (std::exception& e)
  {
    env->ThrowError(
      "MAnalyse: Can not load file Compute.cso %s", e.what()
    ); 
  }

It looks e.what() return only 'ReadData' string.

[magnetite]

Originally Posted by DTL

Same was with magnetite user. Recommended: to put all files in the some very simple folder path like
c:\AVS\
(not something like c:\my_regional_letters_path\some folder Name\avisynth+\ )

Last message he report solve the problem but not exactly how.

I uninstalled Avisynth+ from C:\Avisynth+ and reinstalled it to C:\AVS as suggested, then put everything in C:\AVS\plugins64. I deleted the plugins64+ folder just to be safe. VRAM usage was around 4 GB.

[DTL]

+ some note: as I later found that builds have 'reversed' scaleCSAD adjustment for DX12 processing (2 used as -2 for 'classic MAnalyse' and so on). The default 0 works as 0.
Missed the lines

//                          YV12  YV16   YV24
  // nLogXRatioUV              1      1     0   
  // nLogYRatioUV              1      0     0   
  // effective_chromaSADscales: (shift right chromaSAD)
  // chromaSADscale=0  ->      0      1     2  // default. YV12:no change. YV24: chroma SAD is divided by 4 (shift right 2)
  //               =1  ->     -1      0     1  // YV12: shift right -1 (=left 1, =*2) YV24: divide by 2 (shift right 1)
  //               =2  ->     -2     -1     0  // YV12: shift right -2 (=left 2, =*4) YV24: no change
  effective_chromaSADscale = (2 - (nLogxRatioUV + nLogyRatioUV));
  effective_chromaSADscale -= chromaSADscale; // user parameter to have larger magnitude for chroma SAD
                                              // effective effective_chromaSADscale can be -2..2.
                                              // when chromaSADscale is zero (default), effective_chromaSADscale is 0..2

of PlaneofBlocks.cpp in sending params data to shader.

nLogxRatioUV + nLogyRatioUV for YV12 is 1+1 so
effective_chromaSADscale -= chromaSADscale is
2 - (1+1) = 0 - chromaSADscale - that reverses adjustment.

Will be fixed in the next build.

[takla]

ColorBarsHD(1920, 1080)
ConvertToYV12()
tr = 3
super = MSuper (pel=1, levels=1, chroma=false)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=5, overlap=0, levels=1, chroma=false)
MDegrainN (super, multi_vec, tr, thSAD=150)
Prefetch(12)

Encoded 600 frames in 17.893s (FFV1)
GPU usage is around 8%. Very low.

CPU only takes 14.556s.

[DTL]

It can now work with chroma=true.

super = MSuper (pel=1, levels=1, chroma=true)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=5, overlap=0, levels=1, chroma=true)

The close 'software mode' is only

super = MSuper (pel=1, levels=0, chroma=true)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=1, overlap=0, levels=0, chroma=true)

[takla]

DTL
Using the script posted in #64, optSearchOption=5 is about ~3 seconds slower then optSearchOption=1.

[DTL]

"script posted in #64,"

Do not uses chroma processing in MAnalyse. That may make quality of motion search worse (and cause visible bugs in some cases - https://forum.doom9.org/showthread.p...61#post1783361 ) . So enabling chroma in MAnalyse generally increase quality (and recommended to use if speed is acceptable).

It may be interesting to check the speed lost of MAnalyse between

optSearchOption=1 chroma=true/false (onCPU only processing via host RAM)

and

optSearchOption=5 chroma=true/false (DX12_ME + CS via HWAcc RAM)

The ME hardware engine always uses chroma in search and enable/disable in MAnalyse settings only switch on/off UV planes copying from source to upload buffer and chroma SAD enable/disable compute and addition to output SAD in CS. So with optSearchOption=5 enabling chroma is close to free for speed. And with CPU processing enabling chroma in MAnalyse is typically very visible speed impact.

The second totally winner feature of HW ME - it uses always pel=4 search that is awfully slow onCPU. Though its full usage in mvtools still not implemented (may cause also very slow MDegrain onCPU). It will be used in SO=6 mode for increasing speed with blocksize=16. Using pel=4 precision may also makes degraining a bit better. It possibly can be tested with only optSearchOption=0 or may be 1 max.

"GPU usage is around 8%. Very low."

Is it Video Encode load % of 'GPU (total ?) %' ?

[takla]

It is total GPU load. And I wanted to compare apples to apples, that is why I had chroma set to false for both gpu and cpu tests.
I'm not too worried about quality at this stage, only wanted to test speed. But if the speed penalty for chroma=true is low with gpu, I'll keep that in mind for the future, thanks.

Edit: Tested chroma=true with GPU & CPU, CPU is still faster.

[DTL]

"It is total GPU load."

I hope you use for speed-test release build from github ? Not debug build.

"CPU is still faster."

It looks you use AMD CPU with large cache. For intel-users with low L2/L3 cache the MAnalyse may be more slow.

"Tested chroma=true with GPU & CPU"

What is speed difference of chroma=false and chroma=true for MAnalyse with optSearchOption=5 ? And what GPU-card is used ? With chroma=true in MSuper (so it is enabled for MDegrain).

[takla]

I didn't think debug build would make a difference. My bad. Here are my new tests, again, with the settings used in post #64:

r.2.7.46-a.06
time=15.841s
MSuper & MAnalyse chroma=true
optSearchOption=5

r.2.7.46-a.06
time=15.554s
MSuper & MAnalyse chroma=false
optSearchOption=5

r.2.7.46-a.06
time=4.016s
MSuper & MAnalyse chroma=true
optSearchOption=1

r.2.7.46-a.06
time=3.576s
MSuper & MAnalyse chroma=false
optSearchOption=1

CPU: AMD Ryzen 9 3900x
GPU: AMD Radeon RX 5700

I didn't think the difference would be THAT much. Also, Video Codec usage under GPU in task manager showed 99% usage. So again, my bad on that.

CPU is almost 4 times faster.

[DTL]

"I didn't think debug build would make a difference."

It is full of software checks and very slow. Only provided to get possibly more pointing error message about that C++ exception issue at user-side.

" AMD Radeon RX 5700"

Thank you for information. It is really the first report of non-NVIDIA accelerator usage.

The good news that AMD also provides drivers for DX12_ME operation. And no need for any software tuning between AMD and NVIDIA products. It was expected. Good work for compatibility from Microsoft. The other news is that Ryzen 9 3900x is about 4x faster. Though I have not any info about expected speed of Video Encoder engine in AMD accelerators (if it even exist there). I think it will be some future products from Intel and AMD that will provide hardware features for DX12_ME operation.

Also as expected the speed difference between chroma true and false with accelerator is small enough - about 2%. With 'onCPU' processing is about 12% (still not very large - but it is AMD large-cache device or too small frame size for this cache size).

Update: Known issue and workaround: Lower up to 8 rows of blocks (with block size 8x8) may be somhow not correctly processed at some scenes. Current solution: pad bottom of 1920x1080 frame with 72 lines with AddBorders() and Crop() after processing.
Example:
Addborders(0,0,0,72)
...msuper, manalyse, mdegrain..
Crop(0,0,0,1080)

[takla]

Update: Known issue and workaround: Lower up to 8 rows of blocks (with block size 8x8) may be somhow not correctly processed at some scenes. Current solution: pad bottom of 1920x1080 frame with 72 lines with AddBorders() and Crop() after processing.
Example:
Addborders(0,0,0,72)
...msuper, manalyse, mdegrain..
Crop(0,0,0,1080)

Mhh that is messy. Can't you make mvtools do exactly that, but internally?

[DTL]

It is not long time solution I hope. Just a known way of bugfix *if* it occur. I even not sure where is it come from. Most of buffers copying-processing is totally static but issue happens rarely (may be 1% of frames or less) on some moving patterns near bottom of frame. Like water waves or some camera pan or objects move. May be it even issue of my NVIDIA GTX1060 NVENC (with current driver, etc).
Or it may be SAD compute shader threading dispatch (may not process last threads groups sometime ?) issue. But its Dispatch() call also equal for any frame so the issue should be visible in any frame like no denoising (?) currently if it cause not same blocks blending it may return small or close to zero SAD for several bottom rows of blocks ?

Comment from Microsoft support about used file loading function:

The DX::ReadData helper looks in the current working directory at runtime for the file.

When building for Win32 desktop, if it can't find it there, then it looks in the same folder as the running EXE. If you are a 'plugin' then likely that directory where you have your plug in installed.

Your best option is to use some full path to your "plugin" directory to find your data and not rely on CWD.

So it looks I need to look how 'avstp.dll' is searched for possible paths for more use cases of avisynth + plugins + scripts paths installation and running. It looks ReadData() can not found compute.cso file if mvtools.dll is copied in some path not equal to loaded script path ? I usually put all files in 1 folder (source file, all used plugins, source script) so do not see this issue.

[DTL]

New update: https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.07

Now pel=4 have about usable speed. At i5-9600K + GTX1060 with x264 encoding of 1080i footage SO1 is about 3 times slower in compare with SO5 using pel=4. About 1.26 fps vs 3.76 fps. Pel=1 works with 6.46 fps SO=5.
And it is still onCPU SAD computation for pel 2 and pel 4. Next versions with onHWAcc SAD computation for pel 2 and pel 4 expected to be even faster.
The quality at some quick tested camera pan movements with pel=4 is a bit better in compare with pel=1.

Example of processing script:

tr=25
super=MSuper(last, mt=false, chroma=true, pel=4, hpad=8, vpad=8, levels=1)
multi_vec=MAnalyse (super, multi=true, blksize=8, delta=tr, overlap=0, chroma=true, optSearchOption=5, mt=false, levels=1, scaleCSAD=2)
MDegrainN(last,super, multi_vec, tr, thSAD=250, thSAD2=240, mt=false, wpow=4, thSCD1=350, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16)

MDegrainN new added params:
1. adjSADzeromv (1.0 - default, no op) - possible SAD multiplier for zero-move blocks (before thSAD processing for getting block's weighting value). Float value. Recommended values: 0.9..0.4-. Possible medium values 0.75..0.5. Allow to increase degraining at static areas.

Example: setting 0.5 result thSAD for zero move blocks (static) will be thSAD*2.

2. adjSADcohmv (1.0 - default, no op) - possible SAD multiplier for blocks in coherent moving areas (before thSAD processing). Float value. Recommended values: 0.9..0.4-. Possible medium values 0.75..0.5. Allow to increase degraining at big enough coherent moving areas of much larger 1 block_size size (like camera pan movement over non-changing scene).

3. thCohMV (-1 default, no op) - threshold to detect if block's move vector is equal to surround blocks (top,left,right, down) move vectors. -1 - disables this part of processing (faster), 0 - lowest working value. Recommended values 0..4. Possible range - 0..unlimited int. Too high values will create error-blended blocks (like with too high thSAD value).

The thCohMV is not auto-scaled with setting pel to 2 or 4 and as integer MVs length become 2 and 4 times larger it looks also required correction of thCohMV param about xPel. I do not think internal auto-correction is good because in integer param it will limit range of fine-tuning. Or the integer need to be chenged to float.

Also with pel 2 and pel 4 it looks good to add 1 more param like thZeroMV - threshold to detect if block is close to zero move. With pel > 1 currently even the small sub-pel movements is treated as non-static block. Currently large areas of 'static' blocks also may be adjusted with adjSADcohmv param that can be applied to blocks non-coherency movement (including deltas around zero) defined by thCohMV param.

[takla]

DTL what source filter do you use? I tried both, LWLibavVideoSource and AviSource but they both crash without error.
The only way I can test optSearchOption=5 is with ColorBars.

Anyways, here are some more benchmarks:

ColorBarsHD(1920, 1080)
ConvertToYV12()
Trim(0, 600)
tr = 3
super = MSuper (pel=1, levels=1, chroma=true)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=1, overlap=0, levels=1, chroma=true)
MDegrainN (super, multi_vec, tr, thSAD=150)
Prefetch(12)

r.2.7.46-a.07

ffmpeg -y -benchmark -i TEST.avs -c:v ffv1 TEST.mkv

optSearchOption=5

pel=1 chroma=false
time=14.972s

pel=1 chroma=true
time=15.236s

pel=2 chroma=true
time=15.710s

pel=4 chroma=true
time=15.910s

optSearchOption=1

pel=1 chroma=false
time=3.695s

pel=1 chroma=true
time=4.091s

pel=2 chroma=true
time=4.807s

pel=4 chroma=true
time=5.487s

[DTL]

I use FFMpegSource2() from ffms2.dll. The current builds very critical to V size of frame. Known working sizes 1920*1080 (progressive). 1920*1088 (interlaced for SeparateFields(), 1920*1152 (1080+72) to workaround possible low rows issues and interlaced frame. Still not search in deep what is limit size or how is workaround it inside. But with non-compatible size it throws error like 'error - commandlist-close' or other text error. It looks typical working height (for interlaced source) is integer divisible to 16.

I see you use tr=3 with low params MDegrainN - that cause fallback to MDegrain3 and not tested. It may not work because now MDegrainN uses different way of loading MVs from 'fake' structures. The default creation of 'Fake*' class is to use old scattering of MVs data into FakeBlock class. And will cause crash with MDegrain(X) trying to load from uninitialized memory.
Also SO=5 is not good compatible with MDegrain(X) because their block-procesing functions do not have additional MVs limiting of invalid vectors (sometime coming from hardware ME). With invalid (far outside frame area) MVs the MDegrain(X) will crash without good error message.
Currently only pel=2 and pel=4 with old MAnalyse SAD computation should be compatible with MDegrain(X). The MAnalyse do additional check and limiting of MVs before SAD computing. May be it is good idea to put this limiting into MAnalyse at MVs data coping function from readback buffer to MAnalyse output buffer.

To use MDegrainN with low (any) tr you need to set both thSAD and thSAD2 to different values like 150 and 150-1. To make editing easier it may be created script user-variable like my_thSAD and set it to MDegrainN as

my_thSAD=150
MDegrainN(thSAD=my_thSAD, thSAD2=my_thSAD-1)

Benchmarks shows your CPU is great with core speed and cache size and may be performance of DX12ME at accelerator is limited to 15-sec with tr=3. With 4k frame and larger tr when frameset with pel4 will not fit in CPU cache the onCPU speed may be lower.

[takla]

DTL Specifying a value for thSAD2 is what fixed it, thanks.

LWLibavVideoSource("C:\Users\Admin\Documents\01.mkv")
Trim(0, 600)
Addborders(0,0,0,72)
tr = 8
super = MSuper (pel=4, levels=1, chroma=true)
multi_vec = MAnalyse (super, multi=true, blksize=8, delta=tr, optSearchOption=5, overlap=0, levels=1, chroma=true)
MDegrainN (super, multi_vec, tr, thSAD=150, thSAD2=75)
Crop(0,0,0,1080)
Prefetch(12)

r.2.7.46-a.07

optSearchOption=5

pel=1 chroma=false
time=42.293s
time=41.430s
262.419 KB

pel=1 chroma=true
time=41.253s
time=41.285s
246.503 KB

pel=2 chroma=true
time=45.720s
time=44.713s
248.503 KB

pel=4 chroma=true
time=41.868s
time=44.675s
247.812 KB

optSearchOption=1

pel=1 chroma=false
time=8.963s
265.645 KB

pel=1 chroma=true
time=9.085s
249.453 KB

pel=2 chroma=true
time=14.969s
251.571 KB

pel=4 chroma=true
time=21.300s
248.394 KB

File size was just for me to check if settings were used properly.
optSearchOption=5 seems to sometimes fluctuate by 3 seconds.
optSearchOption=1 is still much faster. I'll test some 4K.

Edit: Here is 600 4K frames with TR=6:
optSearchOption=1 time=205.274s
optSearchOption=5 time=211.290s
Or about ~3FPS.

[Boulder]

I'd like to test the new version, but I'm having a hard time following the complex explanations.

I.e. could someone please simplify things a bit by telling me which settings would make the result as close to a vanilla MVTools2 one as possible, in order to make a fair comparison? My normal chain of operation is MSuper (separate for analysis and MDegrain) - MAnalyse - MRecalculate (possibly two times with halving blocksize with each iteration) - standard MDegrain.

[DTL]

"4K frames with TR=6:
optSearchOption=1 time=205.274s
optSearchOption=5 time=211.290s"

With TR=12 the speed difference may finally reverse to SO=5 is better.

I got an idea how to make at least MDegrainN onCPU better speed for pel >1 still not going into full onHWAcc processing. So onCPU times of MDegrainN part of processing will be lower.
It is about not reading sub-pel shifted copies of frame from memory but shift single block to sub-pel before averaging inside CPU register file or caches. At least for MDegrainN with 1 src block using only 1 ref shifted-position block it should make better speed. And with SO=5 the MVs creation is now serviced with dedicated accelerator. The idea come from the SAD calculation for pel >1 : I not like an idea to upload to accelerator 4 for pel=2 or 16 for pel=4 copies of the ref frames produced by MSuper(). It is slower with upload via bus and wastes fast but limited in size RAM of accelerator awfully. So I will re-make current SAD compute shader to make sub-pel shift of ref block from single full-size ref frame inside shader before SAD calcultaion. It is natural enough task for data compute accelerator.

" My normal chain of operation is MSuper (separate for analysis and MDegrain) - MAnalyse - MRecalculate (possibly two times with halving blocksize with each iteration) - standard MDegrain."

May be better to post your current script so when someone will have time - will tried to change its MAnalyse processing to SO=5 and look if it work / if it possible. Also the input frame size is required.

[Boulder]

Originally Posted by DTL

" My normal chain of operation is MSuper (separate for analysis and MDegrain) - MAnalyse - MRecalculate (possibly two times with halving blocksize with each iteration) - standard MDegrain."

May be better to post your current script so when someone will have time - will tried to change its MAnalyse processing to SO=5 and look if it work / if it possible. Also the input frame size is required.

It's basically something like this. The last MRecalculate iteration is often skipped for 4K so it's just MAnalyse+MRecalculate before MDegrain.

source("whatever.dgi") # 4K or 1080p

blksize = 16 # or 32 for 4K
blksize2 = 8 # or 16 for 4K
blksize3 = 4 # or 8 for 4K
overlap = 8 # or 16 for 4K
overlap2 = 4 # or 8 for 4K
overlap3 = 2 # or 4 for 4K

prefilt = ex_minblur(r=1, uv=3)
superanalyse = prefilt.msuper(pel=4, sharp=1, rfilter=2)
supermdg = msuper(pel=4, levels=1, sharp=1, rfilter=2)

fv1 = manalyse(superanalyse, isb=false, delta=1, blksize=blksize, overlap=overlap, search=4, searchparam=2, pelsearch=2, truemotion=false, dct=5, scalecsad=2, trymany=false, global=true)
bv1 = manalyse(superanalyse, isb=true, delta=1, blksize=blksize, overlap=overlap, search=4, searchparam=2, pelsearch=2, truemotion=false, dct=5, scalecsad=2, trymany=false, global=true)
fv1 = mrecalculate(superanalyse, fv1, thsad=100, blksize=blksize2, overlap=overlap2, search=4, searchparam=2, truemotion=false, dct=5, scalecsad=2)
bv1 = mrecalculate(superanalyse, bv1, thsad=100, blksize=blksize2, overlap=overlap2, search=4, searchparam=2, truemotion=false, dct=5, scalecsad=2)
fv1 = mrecalculate(superanalyse, fv1, thsad=100, blksize=blksize3, overlap=overlap3, search=4, searchparam=2, truemotion=false, dct=5, scalecsad=2)
bv1 = mrecalculate(superanalyse, bv1, thsad=100, blksize=blksize3, overlap=overlap3, search=4, searchparam=2, truemotion=false, dct=5, scalecsad=2)

convertbits(16).mdegrain1(supermdg, bv1, fv1, thsad=150, thsadc=200, thscd1=500, thscd2=90, limit=0.2, limitc=0.45)