New DirectX 12 Video APIs

[Boulder]

Originally Posted by DTL

AVX mean it do not have fast enough integer operations with increased size register file. Also operations are limited to SSE2 128bit integer per op, AVX2 allow 256bit integer ops that is virtually twice faster. It is better to upgrade to AVX2 CPU at least in 202x years. Intel promises AVX1024 in the mid of 202x already.

This.

You can get a very decent second hand AVX2 system on the cheap thanks to AMD. The 5xxx series Ryzens can be used on very old mobos like several X370 or B350 based ones after a BIOS update. The previous generation is also not bad at all, I'm very happy with the 3900X. I'm contemplating on upgrading to 5900X since people seem to be switching to 5800X3D and there's a supply of used CPUs.

[DTL]

Originally Posted by takla

niche cases like TR=>10 with pel=2 or 4 at 4K.

'Big' tr-values really more naturally for 4K and more because with the same amount of light passed via input aperture of camera the samples at 4K tesselation of image receive 4 times less light in compare with 1080. Also if fill-factor of sensors is not great - even less.
So for 4K we need about 4 times more tr to have same relative shot noise in compare with 1080.

Also the new motion vectors low pass processing become more effective with not very small tr values. At least >5..6. So the number of position samples may be covered with impulse kernel of about 10 size. Also if I understand right to make good low-pass filter with cut-off of about 0.1 of the max (Nyquist) band we also need to process not very few samples (may be about 10).

The value noise of samples really cause 'position noise' because the position of image area (patch/block) defined by values of its samples only. So noised values of completely static block start to be translated in the position noise with current motion-search algoriphms (and may be with any motion search algo). After receiving not-zeroed coordinates for completely static block the blending engine start to motion-compensate with errors-noised vectors and it will cause blurring of block. For slow moving blocks it also cause not-perfect motion compensation and blurring. Also blending with shifted blocks with non-zero average value cause some shifting of block in the output frame so performed conversion of value noise in the position noise and MPEG encoder need to find and encode this error-movement too.
The effect of false noise-caused movement may become more visible with decreasing of block size. So the MVLPF processing may be more effective at smaller block sizes (like 8x8 or even 4x4).
So low-pass filtering of motion vectors after search for current pair of src+ref frames in big enough time domain using the nature of real images not have too fast and random moving of blocks can decrease these negative effects. But to perform filtering good enough it need to gather big enough number of motion vectors values of the block in consecutive frames.

The need of creating AVS-interface (script-accessible of samples data for read and write) to the MV data passed from MAnalyse to MDegrain really high. It will allow script-writers to create more advanced algoriphms of intermediate motion search data processing before MDegrain blending in the script form. And after slow experiments the best methods may be encoded into compiled functions of MDegrain or some new compiled filters of mvtools-pack for fast execution.
I think of creating motion-data clip in the RGB16 format with mapping of 16bit (dx,dy,SAD) data to R,G,B channels. But real current motion-data 'frame' have also some header of about 256 bytes inside with analysis data. So need to find if this header can be transfered as metadata unchanged and not in 'active' part of frame.

[takla]

Originally Posted by DTL

'Big' tr-values really more naturally for 4K and more because with the same amount of light passed via input aperture of camera the samples at 4K tesselation of image receive 4 times less light in compare with 1080. Also if fill-factor of sensors is not great - even less.
So for 4K we need about 4 times more tr to have same relative shot noise in compare with 1080.

Oh wow. I didn't think about this but it makes perfect sense.

[takla]

AVX512 is coming you said @DTL?

[DTL]

AVX1024 is coming in mid of 202x if this civilization will not die too fast. I hope it will have even larger register file size to perform more operations in fastest available memory in chip. AMD chips with at least AVX512 support is also good.

From https://clickthis.blog/en/sluhi-o-pr...e-k-2025-godu/


Currently expected the most of performance boost from AVX2/AVX512 operations strategic redesign of MVtools are internal sub-sample shift and scale for both MAnalyse (onCPU processing) and MDegrain (onCPU processing) for pel 2 and 4. It will greatly reduce memory requirement for pre-calculated super clip and it looks very important for new massive multicore chips and current cache AVS design. Also as core become more and more faster in compare with main host memory it should finally make internal scale/shift processing faster in compare with pre-calculated upscaled planes for pel 2 and 4.

I also interested when intel hardware will provide DX12-ME API. It looks intel UHD Graphics 750 still do not have either drivers designed or required hardware features to expose this interface from MPEG encoder.

"I didn't think about this but it makes perfect sense."

Same is about HFR feature of new video systems - the max possible frame accumulation time with HFR is lower so number of accumulated photons per frame per object's view also lower.

So MDegrainN works as 'secondary level video camera' allowing to extend photon's flux accumulation time over the frame's exposure time in 'first level time sampling video camera'. It is only about real optical video cameras (not about digitally synth images). The more image data carriers (photons) accumulated per object's view - the more precision got. The object's view less distorted (noised) by natural photon's shot noise. So the possible tr value is in best case unlimited. Practically may be limited to typical cutscene duration that is several seconds minimum and with even old 24/25 fps it is about many 10'ths or 100'ths.

So the host RAM-based solution allow much higher tr values in compare with full onAccelerator processing (the max tr multiplied to threads number is very limited by accelerator's memory size to hold all required frames). Only sending current+ref pairs to accelerator to ME-process as it work now. Actually it now upload to accelerator current frame once and only send new refs per one output frame of MDegrainN.

[StainlessS]

Methinks you take on monster task DTL, mucho respecto, grassy ass

[DTL]

The more 'monster task' is to add more transforms support in MVtools - like rotate, scale, lighting (additive and multiplied levels change) and may be more. MVtools with only translate transform support are still at the very beginning of development.

[TDS]

Originally Posted by DTL

New version: https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.10

Hi DTL, I had an error with this build with SMDegrain. (In RipBot264)

I can't tell you what the error was other than it was something to do with Line 645, of SMDegrain 3.5.0d

https://github.com/Dogway/Avisynth-S...5.0d.avsi#L645

Or maybe this isn't meant to be used with SMDegrain

[takla]

Originally Posted by DTL

Same is about HFR feature of new video systems - the max possible frame accumulation time with HFR is lower so number of accumulated photons per frame per object's view also lower.

Yep. But the higher frame rate also allows your brain to process more informations. This basically cancels that downside. (When comparred to the same video with a lower frame rate, anyway)

[DTL]

"higher frame rate also allows your brain to process more informations. "

Nowdays before viewing we typically need to very highly compress moving picture data to broadcast or stream via IP or torrents. So MDegrain is mostly part of MPEG compression to make the MPEG output rate and filesize as low as possible with given quality level. I now can make rips from HD1080i sources with x264 crf=18 at medium output rate down to about 4000 kBit/s for low-movement documentaries.

"Line 645"

It is just end of large 'messages output'. Need real AVS error message to see what may be wrong. As pinterf ask it should be compatible with all old scripts when no new params are used (internal defaults should be equal to 2.7.45 release). But to use new features it is required to manually edit scripts like SMDegrain to add new params in no-default values.

[TDS]

Originally Posted by DTL

"Line 645"

It is just end of large 'messages output'. Need real AVS error message to see what may be wrong. As pinterf ask it should be compatible with all old scripts when no new params are used (internal defaults should be equal to 2.7.45 release). But to use new features it is required to manually edit scripts like SMDegrain to add new params in no-default values.

That "message output" IS SMDegrain v3.5.0d !!!

I don't understand what you are saying here, it's a LOT over my head...I would think that Dogway would need to rectify this, so as to use your Mvtools2 build(s).

[kedautinh12]

You need show more detail about error. That not only said line 645. What is error in line 645?? You need sceenshot when error show with ripbot

[DTL]

If your error is with command-line AVSmeter: What is the error text ? Where this 645-line data come from ?

The 645 line at github is 2 }} - this either end of "SHOW PANEL" block or end of all SMDegrain() function. Anyway it is not command or filter or function call line so error may be anywhere above this line (or in any function call from above block).

[TDS]

Originally Posted by kedautinh12

You need show more detail about error. That not only said line 645. What is error in line 645?? You need sceenshot when error show with ripbot

I can't give anymore info, as all AVSMeter displays is the path where SMDegrain v3.5.0d is, and that Line 645 is where the error is, and AVSMeter stops, at that point.

I don't know why it chooses that line which is simply }}, and I guess that's where Dogway needs to have a look.

That's ALL I can do...sorry.

[takla]

@DTL newest release gives me this error which goes away if I delete line 5. Is dithering no longer valid?

[DTL]

Oh - I need to check it. Dithering in AVS ConvertBits() from 16 to 8 - is not in MDegrain. But may be output of MDegrain somehow cause this error. If you feed MDegrain with 16bit it should output 16bit too (with right params or defaults ?) and it can not not perform dithering internally. The DX12-ME mode MAnalyse only can use 8bit YV12 now so you need to make conversion in the script EZdenoise internally if you use 16bit input.

Where is the version of EZdenoise that you use in that script ?

[takla]

@DTL fixed it like this:

function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel", bool "Chroma")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 4)
Pel = default(Pel, 1)
Chroma = default(Chroma, false)

Super = Input.ConvertBits(16).MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super.ConvertBits(8).MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)

Input.ConvertBits(16).MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=thSAD/2, thSADC=thSADC, thSADC2=thSADC/2)
}

[DTL]

I catch this crash with 16bit source: It happens only in release build in MAnalyse in SAD 16bit SSE function. Need some time to find what is wrong. I not touch that part from beginning.

Current workaround may be to feed 8bit to MAnalyse and 16bit to MDegrain.

This looks like working:

function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel", bool "Chroma")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 4)
Pel = default(Pel, 1)
Chroma = default(Chroma, false)

Super = Input.MSuper(Pel=Pel, Chroma=Chroma)
Super8 = Input.ConvertBits(8).MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super8.MAnalyse(Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)

Input.MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=thSAD/2, thSADC=thSADC, thSADC2=thSADC/2)
}

[takla]

@DTL new issues
mvtools2 from printerf MvTools2 2.7.45 and your newest release is no longer bit identical with same settings (post 117)

MvTools2 2.7.45 pinterf
time=21.380s
501.153 KB
ProRes

mvtools2 DTL
time=19.116s
500.510 KB
ProRes

The last build that was bit identical was https://github.com/DTL2020/mvtools/r.../r.2.7.46-a.05

[ChaosKing]

Originally Posted by takla

@DTL new issues
mvtools2 from printerf MvTools2 2.7.45 and your newest release is no longer bit identical with same settings (post 117)

Maybe because some bugs were fixed!?